Portfolio
Computer Vision / AI
ML Inference Optimization
Real-time ML pipeline optimization using TensorRT, batching & parallelization, achieving 10x speedup for computer vision models in production environments. With the help of those optimizations we were able to sell multiple VisionSuite systems running on a single hardware unit while staying within the Python ecosystem.
Face Recognition
Face recognition system developed at Seervision GmbH with a false-positive rate below 5%. By uploading a photo, we can remember talents and always associate the same ID to a specific talent. This way you can track specific people or enable person-specific workflows. This complex and highly requested feature set us apart from other competitors in the market.
Multitask CNN
Architecture for Online 3D Human Pose Estimation and Multi-person Tracking
Robotic systems require precise information about the position and shape of objects along with future motion of surrounding persons in order to efficiently plan trajectories and safely collaborate with humans in a dynamic environment. We present an end-to-end multi-task CNN architecture which predicts object instances, human poses and tracks multiple people. Our method extends Mask R-CNN with a simple tracking head based on the MatchNet metric which relies exclusively on appearance cues. With a straightforward back-tracking algorithm we manage to re-identify lost tracks caused by detector failures or occlusions. Our method runs at ∼2.5Hz and is able to associate objects independent of spatio-temporal relations. Exploiting sensory information from RGB-D cameras and SLAM systems we aim to lay a solid step stone for generating arbitrary 3D human motion datasets and use those to train temporal models to predict human motion for future timestamps.
Does Depth Matter?
RGB-D Instance Segmentation with Mask R-CNN
Robotic systems require precise information about the geometry and the location of object instances for successful interactions with their surrounding environment. This involves the ability to extract information about previously unseen object instances from raw sensor data. Most methods approach this by segmenting RGB, depth, or point cloud data based on either geometrical cues or learned features. Whereas color-based segmentation approaches benefit from distinctive changes in appearance, they fail at identifying objects with similar texture or drastic illumination changes. On the other hand, geometrical approaches suffer from the inability to detect object instances that have invisible boundaries in depth images, such as flat objects. Furthermore, most of the learning-based approaches are restricted to RGB inputs as they require a large amount of training data to show good performance and only limited labeled RGB-D data is available. We extend an established convolutional neural network for instance-aware segmentation to work with RGB-D data, which substantially improves the object instance prediction accuracy, compared to sole RGB inputs. In our approach, we use the synthetic SceneNet RGB-D dataset with data augmentation to generate a large training set and we fine-tune our models using a limited set of labeled real-world data from the NYU Depth V2 dataset. Our method yields real-time performance running at over 11 fps. We evaluate our approach on both synthetic and real data.
IoT
Shop Automation
Self-service in-store system powered by custom smart lockers for oversized products, payment kiosk and Shopify integration. Reduced staff costs to zero (unattended store).
Cloud
Big Data Pipeline
Cloud infrastructure powered by Azure DataBricks, enabling large-scale ETL pipelines, data analytics and machine learning applications.
AWS Cloud Infrastructure
Scalable microservice infrastructure powered by Route 53, S3, EC2, RDS, SNS, CloudFront, Elastic Beanstalk, OpsWorks and ELB. The cloud infrastructure served a web and mobile app enabling a seamless experience at scale.
Robotics
PyJet
NumPy/Python to C++ compiler
The JET library offers a simple way to make Python, and especially NumPy code run faster. This is achieved by transparently converting Python/NumPy operations to performant C++. The computation graph is represented as a Directed Acyclic Graph (DAG). Nodes of the graph represent a specific operation and edges are the data that flows between the nodes. Instead of calling a huge, math-heavy Python/NumPy simulation model on every iteration JET packs the whole model in a single performant C++ function which results in a significant speed-up (up to 60-fold speed up was observed in a simulation of a math-heavy quadcopter-model).
Balancing the Omnicopter
Nonlinear attitude control system
The Omnicopter is a 44 cm×44 cm×44 cm cube-framed UAV with 8 rotors attached on its space diagonals. This work presents the synthesis of control algorithms in order to balance the Omnicopter on both its edges and corners on the floor. Some attitude representations are introduced, with emphasis on the quaternion-parametrisation. The rigid-body dynamics of the inverted 3D pendulum model are derived and a steering system which allows guidance of the balancing maneuvers by the end user is described. The nonlinear attitude control system utilized is presented and an onboard implementation is introduced using the Kalman filter as a sensor fusion algorithm. Performance tests in simulations and experiments are included.
App Development
Trap the Cat
Cross-platform puzzle game built with vanilla JavaScript and CapacitorJS, surpassing 100K downloads across mobile platforms with 100+ levels and user-generated content.
Explora.ch
Cross-platform mobile app built using React Native, providing users with event information and a ticket booking system. Maintained a 5.0-star rating on app stores.
Creative & Entrepreneurial
Beachin' Rentals
Self-service beach & skates equipment rental shop in Barcelona. Clear value proposition with proximity to the beach and 50% cheaper than on the sand.
Eixample Airbnb
Airbnb listing for a flat in Barcelona featuring custom interior design and self-made furniture. Creative expression in physical space creating a unique accommodation experience.
Machine Dreams
An experimental fusion of artificial intelligence and digital art. Surreal animated artwork generated by interpolating selected feature embeddings from a Generative Adversarial Network (GAN).
Anti CryptoPunks
NFT project built on the Polygon blockchain with 6 ETH traded. Grew community to 4k+ users on Discord and Twitter.
