MEL Phoneme System

This MEL script allows the user to enter pronounciations and automatically keyframe each mouth movement to a speech sound, or phoneme, to the mouth of a rigged character in Maya. The animation rig that is being used was developed by Jason Osipa, author of Stop Staring: Facial Modeling and Animation Done Right.

The "Stop Staring" Rig

2D Vector Sliders. Model by Anthony Brown.

The "Stop Staring" Rig is made up of 2D Vector Sliders in the HUD. The sliders correspond to different parts of the face. The values in the sliders are set up like a coordinate system of 2 to 4 quadrants that go from -1 to 1 or 0 to 1 in X and Y space. I'm no animator, so I'm not sure how its attached to the character. My only job was to create a script using this rig that would make it easy for the animator to make the character's mouth move accurately with speech.


The Phoneme Script

When my script is loaded, it brings up a window where the user can type in a series of speech sounds, or phonemes, in the text field. Each phoneme must be seperated by a space. If you want the character to hold a mouth movement for a little while longer, add a space then a hyphen ' - ' after the phoneme. If you want the character to close its mouth after the series is done, add a space, then a period ' . ' . When you are done, set the time increment that you want each mouth movement to set a keyframe. (Default is 5). Once you are done, click "Process Words". The script will automatically set key frames for the Sync, Lips, and Tounge sliders of the rig for each phomene by the given increment.


Example Phoneme Series'

Here are some example phoneme series' to show how to operate the script. After these are keyframed. the animator can tweak the animation to match thier style. If you type in letters that don't match any phoneme included in the script, they will be skipped and give undesired results.

"Hello World" = e l o w - l d .
"Thank You" = th a n k u .
"What the heck are you doing?" = w a th a e k a u - e n .
"This.. is.. Sparta!" = th i s . - i s . - s p a t a .
"Royale with cheese" = r o a l w i th ch e z .

The character saying, "This.. is.. Sparta!"

and "Royale with cheese."

References

I referred to the phoneme lip sync charts in Facial Expressions: A Visual Reference for Artists by Mark Simon, a great book for 2D and 3D character animators.

Phoneme Charts

Procedural Locomotive

For this project I wanted to simulate the movement of a Walschaerts valve gear in Houdini. I knew, before I started, that it was going to involve a good amount of sine and cosine expressions considering the repetitive movement.

Reference Material

Breakdown

Each part of the system is connected procedurally, not physically. Since they are all moving in unison, it just looks like they are connected. Each part is also based on the same global variable, $FREQ, which represents the frequency of movement.

The wheels are simply rotating on the Z axis based on the frame number, $F, and $FREQ. As for the bar connecting the wheels, since it has no axis, and does not rotate, it must be translated in the X and Y axes. To acheive this we need to use a Cosine and Sine function on the X and Y transformations respectively:

tx = $AMP * cos($F*$FREQ+$p)
ty = $AMP * sin($F*$FREQ+$p)

$AMP controls the maximum and minimim distance the Sine curve can reach from zero, the amplitude. So in this case the radius of the circle the object forms when moving. $p is the period of the Sine curve, a shift of where in time the Sine curve is currently read.

The bar in the left movie has Sine curves applied to both the translation and rotation. It is being translated in the X axis back and forth with a simple Sine curve and also rotated in the Z axis with another Sine curve. The pivot point is on the right end of the bar. This simultaneous movement gives the illusion of it rotating with the wheels while driving the crosshead arm shown in the last movie.


+ Links
Walschaerts Valve Gear - Wikipedia

Kitchen at Night

For this project I wanted to simulate light fog coming through window blinds. As the camera moves I wanted to see changes to the light moving through the fog.

Background with no Light Fog, Depth of Field, or Occlusion.

Adding Depth of Field.

To add depth of field, I needed to render out a Z-depth channel. After that, in Shake, I blurred the image based on the Z-depth information.
Adding Light Fog.

To get the light fog pass, I applied a black surface shader to all the objects in the scene. Then I emitted a white light fog from the light beind the window. This created a mask I could use in compositing.

Adding Occlusion.

The occlusion is then added to get the contact shadows.


+ Links

3DRender.com: Lighting Challenges

Flying Cards

This is a Houdini particle simulation. Models of cards are instanced on a particle system. A curve was drawn in the outline of the human figure and moved out in the distance in front. Then one particle is emitted per point in the curve. The particles are then grouped randomly into 5-6 groups. Then the particles in each group are given a random velocity and the groups begin to move at different increments in time. This ensures that the cards will not form the shape of the human in the air before they hit the wall.

Breakdown
There is a random rotation applied to each of the particles as they move. Once they hit the wall, they stop rotating.

Here is a previous clip I created to test things like textures, particle behavior, and motion blur.

A random collision result using Switch SOP.

The card models are given a random texture from these playing card texture files.

Wii Controller in Houdini

For this group project, we are planning on using Nintendo's "Wiimote" as an input device to use inside Houdini. Utilizing CHOPs, we plan on controlling geometries inside the viewport in dynamic or game simulations. Later, we want to capture and record these movements and render them out in a sequence.

List of our Materials

* A PC with Windows XP
* A Wiimote
* Kensington Bluetooth USB Adapter 2.0
* Infared Sensor Bar


Get the Bluetooth Driver

Before even thinking about Houdini, we need to get the Wiimote working on the PC. Depending on the bluetooth dongle you must use specific drivers for this thing to work. Since we are using the Kensington USB 2.0 Adapter, we must use the driver below. So before installing the drivers that are packed in with the dongle, make sure to check which driver you need to use for the Wiimote HERE.

Download the Driver Here: KensingtonBTW_4.0.1.2400.zip

Connect the Wiimote to the PC

After installing the compatible device drivers, do the following:

* Double-click the Bluetooth tray icon.
* Tell it to search for bluetooth devices
* Press and Hold the 1 and 2 buttons on the Wiimote
* It finds the Nintendo Wii Remote (Nintendo RVL-CNT-01)
* Click on the icon labeled "Nintendo RVL-CNT-01" to select it and then clicked "Next"
* Click the button labeled "Skip Pairing"
* "Use a Bluetooth enabled mouse, keyboard, or other interface device" with a "checked" checkbox
* Click Finish


Testing it Out!

After all that, I found a neat little program called WiinRemote on http://onakasuita.org/wii/. It is a Wiimote Driver that basically lets you use the Wiimote as an input device. You can map any keyboard and mouse buttons onto the Wiimote buttons and motions. You can even set up the Nunchuck attachment and use the analog as the cursor and utilize its Z and C buttons! So this is the key. Now that we have got it working on the PC, we can bring it in Houdini!

Download WiinRemote Here: WiinRemote_v2007.1.13.zip


Choosing a WiiMote Software

There are many other software packages that are capable of driving the WiiMote, so search
around or follow the links below, don't limit yourself to this one.

The two main software packages that we used were WiinRemote and GlovePIE:

Advantages of WiinRemote:
-User friendly.
-Good GUI.
-Graphed results.
-Easy to use button configuration.
-Tilt/Roll support for mouse control.
-ZERO coding involved.

Disadvantages of WiinRemote:
-Less control over output options.
-No widescreen/dual-monitor support.

Advantages of GlovePIE:
-Fully programmable.
-Ability to use scripts.
-Visible output variables.
-Configurable output value ranges.
-Widescreen support.
-Everybody likes Pie, and most people like gloves.

Disadvantages of GlovePIE:
-Terrible GUI
-Lackluster documentation
-No Graph representation (tough to
troubleshoot)
-Difficulty connecting to mouse

We ended up using WiinRemote for its easy interface and user friendliness. Now, with WiinRemote running in the background, we launch Houdini 8!


Porting to Houdini

Our basic structure is as follows. We chose to port through the Mouse CHOP, due to our limited knowledge of MIDI's and their outputs. First set the driver to control your cursor. Either pointer if you have an IR sensor, or set it to tilt and roll for axis control. Use a Mouse CHOP in the motion and audio tab. In its parameters Set Activate to While Playing.

In order to deal with the screen edges, make a Math CHOP, and multiply your input variables by your own preferences.

Connect to a Record CHOP, to capture your movements in real time. Finally, connect the Record CHOP to an Export CHOP. In the Export node you can take the X and Y coordinates to any Node variables you wish from SOPs, POPs, DOPs, etc. (Such as transformation, rotation, etc.)


Post-Mortem

In hindsight, the WiiMote was nothing more than a gimmick. We originally thought that due to its ingenuity and unique control scheme it could help revoloutionize how we interact with a three dimensional environments. Due to the current functionalty of the drivers offered for the WiiMote, it has little more functionality than a custom Mouse/Keyboard. Also due to limitations of the drivers it will only accept one keypress at a time, and does not register multiple buttons correctly on the actual remote (nunchuck/power button).

Originally we wanted to use the WiiMote in the DOPS network to control environment/object variables. Houdini however was not designed with real time dynamics simulation in mind with live inputs, and we ran into technical snags at every turn. For example. We used the WiiMote to control a plane's tilt and roll values with a ball dropped on said surface and a gravity force applied. The simulation looked as if it were working correctly until we discovered that if you changed the angle of the plane too violently the sphere would be instantly forced through said surface and we were unable to find a solution. Instead we cheated and used a particle network to recreate a simple ball maze.

Our next attempt was to use the WiiMote as a custom Interface device. We were able to map scripts and keypresses to the existing WiiMote buttons. It worked swimmingly until we tried to pan zoom or Rotate, where we discovered that multiple keypresses were not supported by the drivers we were using. Making complex interactions in Houdini nearly impossible for our purposes.


Conclusion

In conclusion, our WiiMote project was informitave but due to our limited time frame and knowledge of computer peripherals we were unable to create adequate work arounds that would make the WiiMote more usefull or innovative than the standard mouse and keyboard. Perhaps under future builds of drivers the WiiMotes usefulness in 3d environments may be much more substantial. With more knowledge, and understanding of the drivers, we could make "The Tool" for 3d scene navigation and interaction.


+ Links
WiiLi.org
List of Compatible Bluetooth Devices
WiinRemote
YouTube Video - WiiHoudinii (our inspiration)