Field Notes — Dr. Youssef Mohamed

Youssef,
being observed.
& observing back.

يوسف فهمي محمد · باحث في الذكاء الاصطناعي الاجتماعي

Dr. Youssef Fahmy Mohamed spent his PhD teaching robots to respond better to the person in front of them — using thermal imaging and social-context fusion in consented lab studies on effort and frustration signals. Along the way he built ROS4HRI, an open-source framework now running inside every PAL Robotics commercial robot; founded Pokamind, which turned that research into an opt-in communication-skills training product (€43K in Vinnova/Almi grants and €150K in enterprise clients); and keeps a quiet habit of publishing papers people actually cite — six of them, 92 citations and counting.

§1 PhD, Social AI · KTH Royal Institute of Technology, Stockholm (WASP-funded) · 2021 – 2025.

§2 MSc Robotics, University of Bristol · BSc Mechanical Engineering, University of Surrey (Fiat-sponsored thesis).

§3 Founder & CEO: Pokamind AB · Founder: Vizioneer, HighlightsHub · Creator: ROS4HRI · Consultant: many clients, most under NDA — the bulk of the week-to-week work.

§4 Publications in HRI, RO-MAN, IROS, IJSR · 92 citations · h-index 5.

§5 Native Arabic. Bilingual English. Swedish at B1. Fluent in PyTorch.

§6 Currently: Barcelona, via Stockholm, via Bristol, via Guildford, via Cairo.

A brief, honest account
of what Youssef actually does.

A PhD in how robots can respond to the person in front of them. A Master's in how machines can sense the dynamics of a group. Three companies in between. The thread is the same thread it always was: how do we build AI that actually helps the humans it interacts with?

Youssef's doctoral thesis — “Adaptive systems for real-time affective state detection in human-robot interaction using thermal imaging, multimodal fusion, and contextual understanding” — was supported by the Wallenberg AI, Autonomous Systems and Software Program at KTH's Division of Robotics, Perception and Learning. Along the way he published the first automatic frustration detection paper using thermal imaging at ACM/IEEE HRI, then the first context-aware affective fusion paper at RO-MAN, and built the tooling underneath the rest of the field.

That tooling was ROS4HRI — an open-source framework standardising how social robots handle faces, bodies, voices, and the slippery problem of knowing which person is which in a crowded room. PAL Robotics adopted it as the standard HRI toolkit for their entire commercial fleet — ARI, TIAGo, and the receptionist prototype Youssef helped architect as a Visiting Researcher in Barcelona.

Today he channels the same obsession into companies. Pokamind AB is the PhD, productised — €43K in Vinnova/Almi grants, €150K in enterprise clients, and a cross-functional team of 8+ shipping a GCP-backed SaaS that gives each learner structured, opt-in practice on their own communication style — then returns a private, explainable report the learner owns and controls. Vizioneer turns existing cameras in dark stores and cloud kitchens into an operational command centre. HighlightsHub gives every Sunday-league football match its own highlight reel.

Before any of that, a Master's at Bristol (building a system that could tell, from across a room, whether two people were actually enjoying each other's company) and a Fiat-sponsored BSc at Surrey (teaching a light commercial vehicle to steer itself, politely, around pedestrians). The research hasn't really changed since. Only the table size.

Five public builds.
Plenty more behind NDAs.

Each public entry is a working system — not a slide, not a mockup. Shipped, deployed, used by humans and robots. Where possible, click through and try them. The consultancy work, which is most of what he actually ships week-to-week, lives behind signed NDAs — summarised in the final card.

Specimen 01

computer vision · GCP

Vizioneer

AI-powered camera analytics for dark stores & cloud kitchens.

visit site

Vizioneer turns existing camera infrastructure into a real-time operational command centre for rapid commerce. Vision Language Models and classical machine vision work in concert to track orders, optimise picking routes, and detect bottlenecks — from cameras already mounted on the ceiling. No new hardware. Setup measured in minutes. Cameras are used for order and route tracking; the system does not identify, profile, or score staff.

technology stack

Vision Language Models Machine Vision RTSP streaming WebSockets Real-time processing Google Cloud Platform Object detection Tracking

key features

Smart picking routes

Real-time inventory

Order verification

Bottleneck detection

40%

faster picks

99.5%

accuracy

2min

setup

analytics dashboard

dark store optimisation

platform architecture

Specimen 02

LLMs · communication coaching · GCP

Pokamind

Adaptive, opt-in practice for leadership & communication skills.

visit site

Pokamind is a skills-training platform where learners opt in to role-play sessions with an AI counterpart. During each session the system analyses the learner's own communication style — word choices, pacing, clarity, and turn-taking — then returns a structured, explainable summary at the end. Analysis runs per-session; no continuous monitoring, no profiles of third parties, no biometric categorisation. Reports belong to the learner and can be deleted on request. Built GDPR- and EU AI-Act-aware by design, with SHAP/LIME explainability on top of enterprise-grade GCP + Kubernetes (99.9% uptime).

technology stack

Large Language Models Explainable AI (SHAP / LIME) RAG architecture LoRA fine-tuning Self-review of delivery cues (opt-in) Voice pacing / clarity NLP & keyword extraction Kubernetes OAuth 2.0 GDPR-aware design

AI components

Conversational engine

Explainable feedback

Adaptive personalisation

Communication analysis

ROI

68%

faster dev

92%

satisfaction

99.9%

uptime

AI-native learning platform

enterprise security & scale

Specimen 03

computer vision · sports analytics

HighlightsHub

A TV crew for every Sunday league.

visit site

HighlightsHub watches amateur football matches and decides, on its own, when something interesting happened — then stitches together a highlight reel. The system handles goal detection, player tracking, and action recognition through fine-tuned computer vision. Built because the best goal of the weekend shouldn't vanish with the final whistle.

technology stack

Fine-tuned vision models Object tracking Action recognition Event detection Video analytics

core features

Automatic goal detection

Player identification

Highlight generation

Match statistics

auto

detection

live

processing

cloud

architecture

platform interface

Specimen 04

NLP · social robotics · HRI

Furhat · Arabic NLP

A social robot that speaks Saudi Arabic, deployed in NEOM.

Furhat Robotics

A full Arabic NLP stack for Furhat Robotics' social humanoid — tuned for dialectical Arabic and Middle-Eastern cultural context, then deployed at the NEOM Exhibit Center in Saudi Arabia. The robot held conversations with visitors, conducted surveys, and navigated the cultural beats of a Saudi greeting without flinching.

technology stack

Kotlin Java NLP Dialectical Arabic Speech recognition Speech synthesis ROS Cultural adaptation Social robotics

achievements

Arabic language processing

Cultural context adaptation

Natural conversation

NEOM deployment

NEOM

exhibit centre

ar-SA

NLP engine

HRI

social robotics

NEOM is Saudi Arabia's $500 billion futuristic smart-city project under Vision 2030. Furhat Robotics is a Swedish startup building social robots with human-like facial expressions — used in research, healthcare, education, and recruitment worldwide.

Furhat — human-like expressions

Specimen 05

open-source · ROS · HRI infrastructure

ROS4HRI

The framework now running inside every PAL Robotics commercial robot.

view on github

Social robots need to answer a deceptively hard question — who is in the room, and are they the same person they were a second ago? — while merging faces, bodies, voices, and identities from different ROS nodes that never agreed on a common schema. ROS4HRI is the answer: a standard, ROS-native toolkit for social signal processing, co-led with Séverin Lemaignan and presented as the tutorial workshop at IROS 2022 in Kyoto. Adopted by PAL Robotics as the default HRI stack across ARI, TIAGo, and the entire commercial fleet.

stack

ROS / ROS2 Python · C++ OpenFace OpenPose Multi-party tracking Social signal processing Sensor fusion Identity reconciliation

what it solves

Multi-party tracking

Face / body / voice fusion

Stable person identifiers

ROS-native integration

PAL

adopted by

IROS

'22 Kyoto workshop

OSS

MIT licensed

ARI

+ TIAGo fleet

The paper — “ROS for Human-Robot Interaction” (IROS 2021, Mohamed & Lemaignan) — is the reason there's a common language for social signals in ROS today. It is, quietly, the piece of work Youssef is most proud of: a thing researchers cite without knowing whose thesis it came out of.

Specimen 06

consultancy · advisory · confidential

, , & many more

The bulk of the week-to-week work — all under NDA.

under NDA

The five entries above are the public-facing tip of the iceberg. The larger share of what Youssef ships week-to-week is consultancy work across MENA, the EU, and the Gulf — architecting systems for AI startups, guiding enterprises on responsible LLM and vision-model rollouts, and occasionally building the first prototype himself. Most of these engagements are under signed NDA, which is why the interesting parts of this paragraph look like and , and why the client list isn't enumerable here. If one of these descriptions is describing you, feel free to reach out — the rest of the story is available over coffee.

categories he can name

LLM architecture review RAG rollouts CV pipeline design Responsible-AI strategy MLOps on GCP Go-to-market for AI products AI literacy · Career180 (Egypt) E3.Ventures portfolio (KSA) Saudi university workshops

MENA

+ EU + Gulf

NDA

signed × n

∞

can't show

☕

ask over coffee

the mirror test.
try the tech yourself.

A working demo of the open-source face-landmarking tech underneath ROS4HRI — MediaPipe, compiled to WebAssembly and running 100% on-device in your browser. 478 facial landmarks, 52 blendshapes, 30 fps. Smile, raise your eyebrows, or open your mouth wide and watch the page react. Nothing is uploaded or stored. When you close the panel, the camera stops.

What Youssef builds on a Saturday.

When the companies are running on their own for an afternoon, the terminal gets noisy. Local LLMs, VLMs, video generation, facial-expression trackers, communication-style classifiers — things nobody asked for, built because they looked fun. The one below is a Pokamind ad generated end-to-end at his desk: local text-to-image for keyframes, local image-to-video, local voiceover, local music, one ffmpeg command to stitch.

Youssef,
being observed.
& observing back.

A brief, honest account
of what Youssef actually does.