ASTERIX CREATIVE BLOG


Mobile Gyroscope with Javascript and Quaternions Programming Tutorial Part 2

Clay Kent
July 22, 2015


Part 1 covered rotations and Euler Angles. In part two, we are going to explore rotations using the much more versitile quaternion.

Remember the giant catch I spoke about in part 1 regarding Euler angles? Is this the dreaded gimbal lock? Isn’t that the main reason not to use Euler Angles?

To visualize gimbal lock, imagine a gyroscope toy that has all it’s gimbals lined up flat in the same plane like it just came out of the box. In this rare but very plausible state, it actually can not rotate in the 3rd dimension as no gimble is lined up in any way to actually allow that particular rotation. This is called gimble lock. It manifests itself in Euler mathematics as well, making some ‘mathematical situations’ just impossible to do any calculations on without a bunch of “ if soAndSo!=0" etc. etc. So yes, gimbal lock is a huge problem, but in my opnion there’s a actually a worse dealbreaker.

Imagine a piece of pizza lying flat on a table pointing away from you, except magically for our demo purposes, the table can somehow rotate freely in space. Now rotate the pizza 90 degrees on the table’s y axis (pizza still pointing away but pizza side now pointing to your right) and then 90 degrees on the imaginary table’s x axis (pizza pointing into sky – cheese still facing right).

Now, re-position the pizza to the original starting position, but rotate first 90 degrees on the x (pizza point to the sky, cheese facing you) and then 90 degrees on the y (pizza pointing to the right, cheese still facing you)

The raw data is 90 degree rotation on x and 90 degrees rotation on y but the results are different depending on which axis you rotate first. (If you throw in a z axis rotation, it just gets worse.)

On an actual gyroscope toy you will notice the gimbals connect with each other, which means that if you rotate on one of the outer gimbals, the plane of reference for rotation of the more inner gimbals moves along with it. This is called intrinsic rotation as opposed to extrinsic where the axes are fixes. Intrinsic rotation with Euler angles is also called Tait-Bryan rotation. You can remember it because it’s also analogous to roll, pitch and yaw in an airplane: no matter what the aircraft attitude may be. In a nose dive for example, the left rudder causes left yaw (z axis) even though it might not line up with any given plane (z axis or yaw no longer points up for example).

With this being the case, the order in which the rotations happen is critical in order to convey the accurate rotation of the device. It’s analogous to the gimbals on the gyro toy describing the outer gimbals rotation first and the inner gimbals last. This makes intrinsic or Tait-Bryan Euler data really hard to visualize and requires your code to always process the rotations in same order. For the record – the most commonly used order is z then x then y, and that’s the order we use in this tutorial.

Furthermore, the data being generated by the gyro in IOS doesn’t exactly follow this construct as mentioned earlier. While the z axis is always oriented ground to sky, and the x (beta) and y (gamma) rotation data does not always map to anything we can visualize and therefore predictably manipulate with code.

To sum up, trying to imagine what’s going on just by reading 3 rotation values is a great way to give yourself a headache. Surely, there’s got to be a better way.


Introducing the quaternion.

There is a better way to think about 3D rotation than the traditional 3 Euler angles. Imagine tossing a brick out of a multi-story window. The difference in the state that it starts in (let’s say the exact moment it passes the window pane) and the state it ends in (let’s say the split second before impact) can be represented by an amount of rotation on exactly 1 axis (because everything, be it a basketball, planet or falling brick spins on exactly 1 axis – no more and no less) Add to this rotation amount some sort of mathmatical description of what direction that axis actually is actually pointing. To describe this random axis, we use 3 numbers representing the amount that our 3 known axes (x, y and z) are represented in this new axis. For you math types this is simply a 3D vector. To visualize this, we use a 3D Cartesian coordinate system.

With negative values this gets a little weird to think about but it still works. Now add a fourth value, the amount of rotation around this described axis.

For example, a rotation of opening a jar could be represented by a rotation around the z axis of 90 degrees, giving the four resultant values of z:0, x:0, y:1, w(rotation):90.

An almost, but not quite vertical (on the y axis) steering wheel rotation of 45 degrees might be represented by 0, .8, .2, 45

This 4 part rotation representation is called an AxisAngle. It’s often stored in javascript as a 4 part vector or just an array. For our practical purposes, it’s very very close to a quaternion, but the nerd gods of math would be angered if I said they were the same. The quaternion is also a 4 part structure, but has some cool additional properties made possible using the way the values are computed that make things like addition and multiplication (etc.) mathematically feasible (but of course impossible for mortals to read). For example, instead of using 360 degrees or 2*PI radians to describe a full circle quaternions use just use ‘1’. Also it uses normalized unit vectors to describe the rotational axis.

For example: here are some useful normalized quaternions (from the Ogre wiki quaternion primer)

w x y z Description
1 0 0 0 Identity quaternion, no rotation
0 1 0 0 180° turn around X axis
0 0 1 0 180° turn around Y axis
0 0 0 1 180° turn around Z axis
√.5 √.5 0 0 90° rotation around X axis
√.5 0 √.5 0 90° rotation around Y axis
√.5 0 0 √.5 90° rotation around Z axis
√.5 -√.5 0 0 -90° rotation around X axis
√.5 0 -√.5 0 -90° rotation around Y axis
√.5 0 0 -√.5 -90° rotation around Z axis

[w, x, y, z] = [cos(a/2), sin(a/2) * nx, sin(a/2)* ny, sin(a/2) * nz]

None of this makes sense? Don’t worry, it doesn’t have to. As lazy programmers, we don’t necessarily need to worry about a quaternions inner structure, as we just need to know that we can convert to and from easy-to-use AxisAngles and their extremely useful equivalents (aka the quaternion) quick and easy.

The wonderful thing about quaternions is that if we want to combine 2 or more rotations, all we have to do is multiply the quaternions together. This means that combining all sorts of rotations is as easy as: qRot1 * qRot2 * qRot3. We can also take any point (a 3D vector) multiply it times a quaternion and get it’s resultant position. This is the same functionality as using a matrix to modify a point in space but it’s a lot more intuitive.

As with all goofy math things, there is a catch. Remember the order issue described earlier? The same holds true with quaternions. Unlike regular multiplication where order doesn’t matter, with quaternions, multiplication must be done in order. But not just any order. You must go backwards. That is to say, the last rotation goes first and the first changes come last. Changing the order will give you strange results. The technical term for this is that quaternion multiplication is non-commutative.

For more theoretical info on quaternions check out:
Wiki Quaternions and Spacial Rotation
Wiki Quaternions

Enough talk, Let’s replace the Euler logic from the 1st part with quaternion functionality.

first let’s remove a little bit of code will no longer need: take out gobal variable definition:

var deviceOrientationData ={alpha:0,beta:0,gamma:0};

and remove from processGyro:

deviceOrientationData.alpha=alpha;
deviceOrientationData.beta=beta;
deviceOrientationData.gamma=gamma;

Now,lets add some display html so we can see the quternion values in real time

and the stylesheet info to display it in the lower right corner

add a javascript global to store the gyro quaternion

now add the function that creates a quaternion from an AxisAngle:

Note that we could actually use this function to generate 3 separate qauternions, one from each axis of rotation, and then combine them with multiplication (function described later) to create a master quaternion. The code would look like:

But there’s a faster way so we won’t use this code.

Instead we create a function that creates a quaternion from 3 rotations simultanesouly:

First add the javascript to compute the quaternion. In the processGyro function add:

create the computeQuaternionFromEulers function and a simple quaternion object making utility funtion to save on typing

Let’s go back to the processGyro function and throw some optional non necesarry test data on the screen using the numerical values of the quaternion. Put this code in at the end of the function.

Now to make it render using this data. First, goto renderobj() and change the first line from:
var rotatedObj=rotateObject(obj);
to:

Were adding an extra argument to the rotateObject function, a quaternion to represent the rotation. However, as pointed out before, we don’t actually want to render the object rotated the exact same as the device. A traditional gyroscrope resists movement no matter how you orient its base. For our virtual gyro on a mobile device screen to give the same effect, it needs to render the exact opposite from the gyro movement info it is producing. Luckily for us, making a quaternion rotate in the opposite direction is extremely simple. Simply mutiply the ‘w’ componant by -1. Let’s create the inverseQuaternion function to do just that.

As you can see, there’s almost nothing to it. All the function does is spit back a new object with the exact same values except w is negated. That’s it.

Next, back to the rotateObject function to make it deal with the new second quaternion argument. There, we change the declaration to deal with the extra argument:

and then we change one line from:
newObj.vertices.push(rotatePointViaGyroEulars(obj.vertices[i]));
to:

Were simply changing the method used to rotate the point and passing the quaternion along as an extra argument. Lastly, it’s time to replace the meat and potatos of the code. Remove the rotatePointViaGyroEulars function altogether as we’re no longer using Euler data, then add the rotatePointViaQuaternion function

Compare that to rotatePointViaGyroEulars. It’s a bit cleaner. But what’s going on? Quite simply elegantly it turns out that if you multiply a quaternion and a 3D point together you simply rotate the point. Well almost, you technically need to multiply the quaternion and the point and then the quaternions conjugate. The conjugate value of the quaternion is simply the same thing with the x,y and z values negated. ( For you math types, multiplying by the conjugate removes the non real i,j and k values from the quaternion) The embedded conjugateQuot funtion takes care of doing this. Fascinatingly enough, instead of using the conjugate you could use the inverse via the inverseQuaternion(gyro) function and get the exact same effect. Try it, it works. That’s seems to indicate that {-x,-y,-z,w} = {x,y,z,-w} –someone correct me if I’m wrong. Anyway, I’ve left it with the conjugate for completeness as I’m sure the math nerd mafia would point out a difference but either way works. If you’re still awake and paying attention, you might be wondering how a 3d point was grafted into an equation between quaternions. This is also very simple, the quaternion version of a 3d point is simply the x,y and z values of the point plus a value of 0 for the w value. Note that this resultant quternion has no rotation and is most likely not a unit value (aka circumfrince of radius defined by point does not = 1) like quaternions are supposed to be, but the math still works just fine. Technically quaternions should be always be normalized to unit value, but in reality it seems to make no difference (at least in javascript) so I’m leaving it out all together(and it’s processess hogging sqaure root). For those of you who are curious the normalizing function would be (again not neccesarry in this tutorial):

Anyway, back to the point rotation function. This function calls quaternionMultiply() which is perhaps the most useful thing about using quaternions. You can combine any number of quaternion rotations by multiplying them together. However, multiplication with quaternions is non-commutative. That means order is important. The function quaternionMultiply() takes an array of 2 or more quaternions, multiplies them together, and returns a new quaternion object. Of course there’s one last idiosyncracy, the last quaternion argument is the ‘start’ and the first quaternion argument is the last change you to want to compute (even though technically the function calculates from left to right) So to summarize: you can chain multiple quaternion rotations (and even throw in a point) to get the final rendered point, just remember to order the data backwards to what you would intuitively think it should be.

Add the code for that function:

All done! Here’s the final code. Load this up and take a look.

All this quaternion nonsense seems like a lot of extra work for little benefit. It looks like we just swapped out one bit of unintelligible code for another. So far in this example, this is somewhat. But what if we wanted to layer multiple rotations on top of each other, a very common situation in dealing with anything 3D, this is where quaternions shine.

So let’s make this interactive. Were going to spin the cube by using our finger. First, let’s add some html to show some more test data. Add this between the quatInfo div and the canvas tag:

and the css to put in in the lower left corner:

Next lets add some javascript globals to store user interaction data:

Next, add some code to get the finger touch events (with limited backwards compatibility to mouse interaction):

All this code is to gather x and y locations for finger events. The move events forward off to the userXYmove(x,y) function to actually do all the work. The start and stop events will become useful later for storing and resetting delta values but will get to that later. Add the code for userXYmove:

This code fires on any movement event. It creates 2 quaternions. One from horizontal finger movement to create y rotation values (note this relationship is non intuitively opposite from what you might first think) and another from vertical finger movement to indicate x axis rotation. These 2 seperate axis specific qauternions are multiplied together to form a new quaternion representing both movements together. This is quaternion is stored and used to render the cube.

To make the code render this new info, go back to the renderloop() function and change the section of code that reads:

renderObj(cube);
renderObj(xAxis);
renderObj(yAxis);
renderObj(zAxis);

to :

We’re adding an extra parameter here to send to renderObj(). The new parameter is the quaternion describing how we want to rotate the object for render. The three axes get the standard revered gyroscope quaternion, but ‘cube’ gets userQuat combined with (mutiplied by) the inverse gyro. Now let’s go back and update renderObj funtion code. Change the declaration and first line of the function from:

function renderObj(obj)//renders an object as a series of triangles
{
  var rotatedObj=rotateObject(obj, inverseQuaternion(gyro));

to:

Now go ahead and try it out on your phone. (Here’s the code so far)

That’s something, but it seems to jump a bit. Lets Try and smooth it out some. Replace the last 3 lines of userXYmove with the following

Now the code uses those flags we set up earlier to differentiate between different finger swipe events. This prevents all the jumping around. Furthuremore, now were storing previous data to measure changes in x and y movement and using that to augment the existing user control data. Take a look at how this performs:

That’s nice, much better but it’s only about to the point you would get if you enabled OrbitControls.js or something like that, but you’ll notice it’s still a bit unpredictable. While x and y drags on the screen do change x and y rotations, the cube is already rotating so the x and y axes no longer line up with the users x and y perspective giving unintuitive results. Let’s actually give the user the user experience they want and make it react exactly as they would think it should. To do this, we need to map the users input data over the already rendered gyro data on the fly. Luckily this is fairly easy to implement with quaternions. Change this line in userXYMove:

userQuat=quaternionMultiply([yMovementQuat,xMovementQuat,userQuat]);//use reverse order

to read:

Now the processing order starts with the existing user quaternion, adds the inverted gyro (how the scene is rendered), maps the x then the y changes (since they relate at any given point to the rendered inverted gyro), and then finally removes the inverted gyro offset by multiplying back in the actual gyro. Give this a try. (And check out the source for what the final code should look like)

There we go, nice smooth movements that intuitively react exactly the way you think it’s supposed to. Sort of like an the trackball controller on the old missle command console game. Easy, simple and practically explains itself. I rarely see implementations get this right, but without it you’re missing a whole world of 3d interactions. By implementing this you’re not just getting x/y data, you have rotational state plus x/y data opening up a world of possibilities. For example swiping left to right with the phone flat could have entirely different funtionality than swiping left to right while holding it vertical. On a personal rant note, I rarely see any 3D rotation controls that work like this. People just don’t care or can’t figure out how to do it. It’s ‘interesting’ to note that when you implement something really intuitively and seamlessly, people just assume there was nothing to it, or that it was already built into the OS somehow. Package all this up into a nice library and I suppose it would be. Anyways, Merry Christmas Internet.


Coming soon: part 3: Using WebGl to render gyro data


That’s it. Were all done. Now take this info, make something in javascript or port it to whatever language you desire, and make something interesting. If you do, I’d love to hear about it so send me a line. Thanks for reading!!!


BTW, If you want to see a version of this in three.js look here.

P.S. Shamelss plug: If you’d like to see an IOS app that I wrote which uses all these 3D principals, check out 3D gyro draw or Sketch and Etch.

Leave a Reply

Your email address will not be published. Required fields are marked *