Portfolio

At Fraunhofer MEVIS, I spearheaded the continuous development of the on-premise Nomad cluster and the surrounding tool landscape, including Vault and Consul servers, but also custom web applications to allow easy scheduling of common deep learning tasks. Additionally, I conceptuatlized and implemented our infrastructure on top of a managed on-premise OpenStack in close collaboration with our IT, the external service provider, and internal stakeholders.

MeVisLab is an editor and framework to prototype image analysis algorithms. It focuses on medical image processing and includes a node-based editor to quickly evaluate new ideas and create specialized standalone applications. It is mainly developed by MeVis Medical Solutions and Fraunhofer MEVIS contributes modules as well. At Fraunhofer MEVIS, I took part in its development: My main responsibility was the integration of third-party libraries in C++ and Python. I also maintained parts of the build infrastructure at Fraunhofer MEVIS, especially around Conan, which is used to build and cache third-party artifacts. Additionally, I supported users with CMake-related questions and introduced SBOMs for our third-party software.

De(a)d jokes is a comedian's nightmare: He needs to find a good joke for an ever demanding audience. I got a chance to learn how to create a pixelated look and feel using a combination of low and high resolution cameras.

I built an AR treasure hunt game as a birthday gift for a "Divinity: Original Sin 2" fan. The game features Lohse helping the heroines to level up by training their heroic skills. The group of heroines solved puzzles and challenges along the two-hour treasure hunt. To work, the game also included a real "potion box" filled with energy drinks, some party hats for armor, a box with chocolate bars as rations, and a dinosaur piñata as the final boss.

For a seminar, I implemented a small game engine and game. In Robowars, the participants programmed their own bot in Python to learn and discuss, among other things: Python packaging, containerization, CI/CD, and running containers on Nomad. In order to handle many robots, I replaced Python's xmlrpc library with an asynchronous alternative I implemented with Starlette. The frontend is written with Svelte.

In this game, the main character needs to take care of a tree by gathering water and food. Each item required to take care of the tree can be gathered with a different mini-game, but the tree becomes more demanding over time. I took care of the scene changes and keeping track of the game state, leveraging Unity's SceneLoader.

I built the DLU pipeline as part of the EASE project. It takes a natural language command and a scene as an input, parses the utterance into a semantic specification, and uses ontological knowledge and previous experience to create a task plan. This task plan can be simulated before a real robot executes it.
S. Höffner, R. Porzel, M. M. Hedblom, M. Pomarlan, V. S. Cangalovic, J. Pfau, J. Bateman, R. Malaka (2022): Deep Understanding of Everyday Activity Commands for Household Robots. Semantic Web, IOS Press.

I created a portfolio in the style of a classic newspaper highlighting some of my achievements and awards in the past. It is now outdated, but I kept it online.

SOMA is an ontology for everyday activities based on the DUL upper model. It extends DUL with different Event types, e.g., States and Accidents, and also contains concepts of typical household items in its extensions. I used SOMA as the basis for DLU.
D. Beßler, R. Porzel, M. Pomarlan, A. Vyas, S. Höffner, M. Beetz, R. Malaka, J. Bateman (2021): Foundations of the Socio-physical Model of Activities (SOMA) for Autonomous Robotic Agents. FOIS 2021.

In this multiplayer game, you play a witch or wizard apprentice at the end of their wizard training. But to become part of the inner circle of witches and wizards, you need to survive one last challenge: losing yourself to find wizard wisdom. With each successful task, the apprentices lose one of their five abilities, so that they must cooperate to benefit from all.

In this study we investigated whether taking people out of their virtual experience causes a break in presence, which would eventually lead to different results in questionnaires. We measured skin conductance as a proxy for participant stress and were able to find changes when participants were asked to take off their VR headset. This is an indicator that it makes sense to administer questionnaires in VR instead of outside.
S. Putze, D. Alexandrovsky, F. Putze, S. Höffner, J. D. Smeddinck, R. Malaka (2020): Breaking The Experience: Effects of Questionnaires in VR User Studies. CHI '20, ACM.

We investigated different ways of posing questionnaires in VR. We compared different design choices, for example, using a controller similar to laser pointer and other things. To create less biased results, we created a game with a different input modality to be played before answering the questionnaires, in which the players got immersed by shooting some balloons with arrows.
D. Alexandrovsky, S. Putze, M. Bonfert, S. Höffner, P. Michelmann, D. Wenig, R. Malaka, J. D. Smeddinck (2020): Examining Design Choices of Questionnaires in VR User Studies. CHI '20, ACM.

Because the Machine Learning world flocked to Python (and the university's MATLAB licenses were not renewed), we translated old exercises from MATLAB to Python and created new and updated exercises for the Machine Learning class. The exercises range from custom implementations of decision trees to multi-layer perceptrons. My favorite implementation are the self-organizing maps as they are a powerful tool to identify similarities in data.

To allow students to implement Computer Vision algorithms in a more interactive and fun way than on example images or videos, I developed cvloop, a tool to interact with OpenCV video streams from within Jupyter Notebooks. It abstracts away all the frame grabbing and allows to apply functions on the video images – on the whole image or only some parts of it.

In MEANinGS, we built a VR game to gather behavior data in a kitchen environment. I used some of the data to create ontological knowledge from experience in DLU.
J. Pfau, R. Porzel, M. Pomarlan, V. S. Cangalovic, S. Grudpan, S. Höffner, J. Bateman, R. Malaka (2019): Give MEANinGS to Robots with Kitchen Clash: A VR Human Computation Serious Game for World Knowledge Accumulation. ICEC-JCSG 2019, Springer.

For one of my projects, I implemented a plugin in Unity to allow objects to follow trajectories. To use it, one specifies poses along a trajectory using Unity GameObjects and attaches a script to the object which should follow. The trajectory is interpolated between the points using a Hermite spline. I even handled ease-in/-out and a switch between physics or not. Even though it worked for me, let me kindly suggest to use iTween instead.

For my master's thesis, I attempted to track gaze only using a laptop camera. Eventually, I was able to track eye centers and whether a persons looks roughly at the upper or lower half of the screen. The center detection is based on an implementation by Tristan Hume. For the gaze estimation, I assumed an idealized head model based on medical measurements and mapped faces onto it using dlib's face landmark detection models.

In this project, we aimed to reimplement two papers about depth estimation from monocular images on the Sun Grid Engine, a process scheduler similar to slurm. We used the parameter server of TensorFlow to allow multiple worker nodes to train simultaneously and share their weights.

During my master's degree I taught a class on Python programming. The lecture was tailored towards students who started their master's program but had no previous training in programming. Other students also joined as Python was not taught in the Computer Science classes but very relevant for classes such as Machine Learning or Computer Vision. I created all lecture materials, delivered the lecture, and offered a weekly walk-in practice session to assist in the homework exercises. My teaching performance was rated among the top 10% at the university in that year.

My first major professional project was Wasabi at Intuit. As part of the Intuit Data Engineering and Analytics team, I implemented several features in the A/B testing service, including experiment bookmarking, audit logs, and data versioning for the ETL pipeline.

The lecture PMPC had no script available. So we set out to share our notes and exercise solutions – the result was an almost complete script of the full semester, including the exercises and solutions. This was a perfect opportunity to improve not only our modeling and probability theory skills, but also our LaTeX and writing ability.

In my bachelor's thesis, I implemented a particle filter for probabilistic robot localization in 3D. To localize itself in a known map, the robot uses a laser scanner and compares sampled poses in the map with its sensor data to select the most likely candidate. The algorithm is based on the classical AMCL2D. I wrote the implementation in C++ for ROS.

The fire particle simulation we built in the parallel algorithms practical course directs particles with low-pressure areas to form a cone-like structure. It also features heat flicker, causing distortions above the visible flame.

In the practical course for computer graphics, my team received the positions and velocities of simulated water particles in a terrain. We used the terrain and particle information to render a water surface using deferred shading.