The vvvv multipurpose toolkit is a visual programming environment for easy prototyping and development. Changes in the program are applied live and and it can be extended by using the provided plugin interface. It is designed to handle large media environments, as well as real-time motion graphics, audio and video.
Throughout the past years, I have been contacted by customers several times asking for developing plugins for the vvvv multipurpose toolkit. At this point I would like to present a short summary about some of the plugins I designed and implemented using C# and the .NET framework.
Reactivision Interface
Reactivision is a open-source and cross-platform computer vision framework for fiducial marker tracking as well as for finger tracking on multi-touch screens. We extensively used this framework for the position tracking system on the Responsive Light Bots project. One of my sub-tasks was to extend the C++ Reactivision sources in order to make the framework capable of calculating the Robots actual position on the basis of recurring fiducial marker patterns. I designed and implemented a position calculation algorithm and a bi-directional inter-process-communication interface to interchange position data and camera control messages between the robots and the central control system. The positioning system achieved a precision of about 2mm, depending of the infrared camera resolution used.
ORB-SLAM Interface
Simultaneous mapping and localisation (SLAM) systems are in the field of robotics used to construct and update a map of an unknown environment while simultaneously keeping track of the agent’s estimated position and orientation. We experimented with different sets of sensors, including monocular cameras, stereo cameras and depth sensors like the Kinect or Astra sensors. Besides evaluating different sensor types, my job was to compile the C++ sources and create another vvvv plugin that interchanges map data and control information between the SLAM system and the vvvv multipurpose toolkit. I designed and implemented an network socket based inter-process-communication interface. The advantage of network socket based IPC is that the processing power hungry SLAM system may run on a different computer than the high-level control algorithms.
MAV-Link Ground Control Station (GCS)
MAV-Link is a lightweight communication protocol to control unmanned aerial vehicles (UAV), like drones and airplanes. It is also used to communicate between onboard hardware components. Messages are defined within XML files and are distributed by a publisher-subscriber and point-to-point design pattern to their corresponding receivers. I designed and implemented a set of vvvv plugins with C#/.NET that fully wraps the entire communication protocol features. This allows the user to control any MAV-Link UAV by using the vvvv multipurpose toolkit. This project is powered by Colorsound IXD.
Crazyflie Ground Control Station (GCS)
The Crazyflie is a small, versatile and highly expandable quadcopter and is controlled it’s own communication protocol using the nRF51822 radio chip in it’s basic layout. Bitcraze provides a library written in Python that handles the low-level communication between the drones and the ground control station. I developed a complete set of vvvv plugins to remotely control the drones and receiving the sensory system’s data. This project is powered by Colorsound IXD.
Moncha Show Laser Controller
This project aimed to implement a vvvv plugin to communicate with the Showtacle’s Moncha laser controller hardware. This time, I solved the bi-directional communication between vvvv and the laser controller’s C++ DLL by P/Invoke technology. This allows to access structs, callbacks and functions in unmanaged libraries from managed C# code. This project is powered by Colorsound IXD.
