During weekend Klemens and me worked hard on a Jnect-M2M integration.
If you follow the image from “Sharky – his evolution progress“, you can see that OpenNI and NiTE are running on an Ubuntu hardware and not on a beagle bone black. The problem was, that NiTE does not support ARM processors for now.
Ubuntu running OpenNI and NiTE
So we got the idea to track the coordinates of parts of the human body (joints) on the Ubuntu hardware using OpenNI and to send the coordinates to an external M2M-Server running on a beagle bone.
The informations sent by OpenNI java wrapper to the M2M-Server (topic=skeleton) look like:
skeleton {
joint = Left-Hand
x = 123.45
y = 211.77
z = 86.78
}
Jnect with M2M Client
We used the Jnect project and added M2M support to it. So Jnect no longer depends on the Microsoft Kinect Library, but also may use a M2M-Connector to get skeleton information from the M2M-Server.
Jnect will subscribe to the topic=skeleton from the M2M-Server and gets the coordinates of the tracked joints by OpenNI. Implementing some glue code it was simple to build the EMF-body-model defined by Jnect. Since Jnect also provides API to register GestureDetectors, we could use the provided API to add a LeftHandUpGestureDetector.
Finally we installed the Jnect bundles on an Equinox OSGi runtime running on a beagle bone black.
What happens in detail
The ASUS 3D-sensor is connected to the Ubuntu hardware. Running an implemented Java-Wrapper, the sensor will send pictures to OpenNI and OpenNI will pass skeleton informations to the Java-Wrapper. We put the given coordinates to a datastructure and pass them to the M2M server using Eclipse Paho.
The M2M server receives the messages and passes it to the Jnect M2M client running on the beagle bone black.
Jnect will parse that information and adjusts the EMF-body model. Changes to the body model will invoke the LeftHandUpGestureDetector. If the gesture is matched by the changes of the coordinates sent from OpenNI, then a systemOut is sent to the console.
Flos Blog 