That demo only read the position of the hand and moved the robot arm up and down to try to mimic the hand movement.
We have now improved on that to include full Upper Body Tracking.
This means the Robobuilder should be able to much closer mimic the position of arms by tracking shoulder, elbow and wrist (within it's own DoF limitations):
An important heads up is that this code is designed for Kinect SDK v1.0
How we've implemented it:
WARNING! Math contents ahead!
A human shoulder has 3 degrees of freedom, meaning it can move in 3 different ways. Pitch, roll and yaw. A human elbow has one degree of freedom, pitch. More info here.
Our objective was to try to copy those movements into the Robuilder arm. However the Robobuilder arm does not have shoulder roll. That meant we had to sacrifice one degree of freedom.
To those more familiar with these subjects, what we are doing here is not inverse kinematics . We do not want to move the robot hand to a specific position in space, but rather move the robot arm in a similar way to the human movement. This means we only need to work with angles.
The Kinect outputs joints positions in a coordinate system present at the end of this site.
Since we need to get two angles for the two shoulder servos, the use of spherical coordinate system seems adequate.
The steps performed for each side of the body are:
The 3d position of the elbow is subtracted from the 3d position of the shoulder.
This gives a 3d vector corresponding to the upper arm angle in relation to the body.
That coordinate system is transformed into a coordinate system, where the Z axis points from the shoulder joint to the outward direction.
This enables us to obtain the azimuth and inclination angle by using these formulas.
(All of this assumes the person is facing the Kinect, so some strange results may occur if this is not the case)
For the elbow angle another approach is used:
By subtracting the wrist position from the elbow position, we get the vector corresponding to the forearm.
We can then use this method to obtain the angle that is used for the elbow.
Because of the limitation on the degrees of freedom only the elbow angle is copied, and not the forearm direction. This means that in some cases the movement is not correct.
Still we think this works out nicely.
Also visible in the video of another demo we prepared, where the Kinect is rotating to track a person.
I did the math for this, so if you have any questions let me know.
Pedro did most of the programming, so he will follow up with details on that
Regards
Last edited by MarcoP on Fri Jun 22, 2012 10:51 am; edited 1 time in total
What a great update , what has been published shall return all the programs used to get it? or you still working on the same versions as in the previous work with Kinect?
As usual some of the work was facilitated thanks to your libraries for Robobuilder.
It is the standard Kinect and we will share the code as before. Since it was Pedro that wrote it, i better leave that for him. I suppose it will be done in a few days.
Sorry for the time it took to get back to you with the source code.
Time runs fast over here! (Marco had to give me a heads up to remind me we hadn't published the code)
Before diving into the code it is important to understand the major improvement in this version is full Upper Body Tracking.
We're now tracking a lot more joints: Should, Elbow and Hand and calculating the vectors between each pair of joints in 3D space.
We then map the angles to actual Servo positions (within some pre established boundaries as the Robobuilder DoFs are much more limited than ours). Marco explains this is great in his first post of this thread.
An important heads up is that this code is designed for Kinect SDK v1.0
I've also re posted this information on the top of the thread to make sure people find this easily.
We haven't implemented the cool Kinect Speech Recognition feature that l3v3rz uses as these demos are for use in Trade shows where the ambient noise is too loud. Having said this if anyone wants to add it to the code, we'd more than happy to re post it.
More interesting new features:
On the GUI you'll notice an additional row with new fields at the bottom of the Window:
This is a new functionality that we've added to Track the Person in real time by rotating the Kinect on a motorized platform.
(you can see this happening in the videos on the first post).
If you supply a "Stepper COMM" and click connect, the software will start sending serial commands regularly over the COMM port with the actual Angle of the person in relation to the Kinect (displaye din the field "Kinect Angle").
This is done by calculating the angle between the center of the Kinect Space and the person's Head.
In our setup our Kinect was on a Rotating Platform controlled by Arduino and we kept sending angles to the platform to continuously keep track of the person.
If you want to implement your own solution the protocol is quite simple:
2 bytes:
- first byte is an "R"
- second byte is a signed byte the actual angle. (fyi a Signed Byte is still a byte but ranging from -127 to +127 but it's still 8 bit).
The positive/negative give you the direction (L/R).
If the person becomes out of range, you'll get an angle of 127, meaning person lost.
You can use this information to continue tracking for people of just stop.
To prevent shaking and overshooting, there is also a hard coded dead zone; I believe nothing is sent if the person is within +-5º of the center.
We hope you give it a try and let us know your thoughts on the improvements
Yes that is correct.
If you already have all the dependencies installed (i.e. if you already ran the previous version) you shouldn't need to install anything else.
Just download the new code and run it.
We haven't changed or added any dependencies on this new version.
I must be doing something wrong because I downloaded the new version but I can not make it work, the program is not responding.
The application I try to run this in the folder C: \Robobuilder_V2_ SkeletalTrackingVB_SDKv1 \ Robobuilder_V2_ SkeletalTrackingVB_SDKv1 \ SkeletalTracking \ bin \ Release
Forgive my English level, but I am using the google translator.
I open the program and the balls move, put the port number of kinect and the program says that this "connected", but when I was connecting its port robobuilder corresponding, the program gets stuck and I can not close or do nothing.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
£22.01 inc. VAT
, what has been published shall return all the programs used to get it? or you still working on the same versions as in the previous work with Kinect?
