Tld-based Musical Instrument

Below is a demo of a gtk application I’m writing that uses computer vision to create a musical instrument.  The instrument consists of shapes drawn over video captured from a webcam, each shape representing a note.  Computer vision algorithms analyze the video to detect an object of our choosing, usually the hand of a player.  Moving the detected object into a shape sends a midi note to a software synthesizer, in this case, yoshimi.  Audio and midi is processed with using the jack connection toolkit.  The video was recorded using ffmpeg libraries.  I threw in a spectrogram for fun so I can claim I’ve written code that uses fftw :).

Eq Interface

Screenshot from 2014-12-21 06:00:59vlcsnap-2014-12-21-06h13m44s242Screenshot from 2014-12-21 05:49:55


Krad Radio Web Real Time Compositor Control Demos


GNOME Shell Window Focus Colorful Clutter Animations

I learn something every time I try to make the GNOME Shell focus-effect extension more obnoxious. This time, I eased multiple properties in one Tween, just like a pro might have done. And, can you say, Math.random() colors?

Below, find two screen capture demos of making stuff happen with clutter actors and so forth when windows take focus. Believe me, I’ve got more obnoxious ideas for this extension.



gstreamer + v4l2loopback == x11 as a Web Cam


  1. Get and install the v4l2loopback device
  2. $ sudo modprobe v4l2loopback devices=2
  3. $ gst-launch-0.10 ximagesrc use-damage=0 ! ffmpegcolorspace ! “video/x-raw-yuv,width=1920,height=1080,framerate=30/1″ ! v4l2sink device=”/dev/video3”
v4l2loopback device fed streamcapture by gstreamer, v4l2loopback device is used by WebRTC and captured and displayed in wayland by krad radio
v4l2loopback device fed screen capture by gstreamer; v4l2loopback device used by WebRTC and krad radio, the later captures and displays in wayland window
Screenshot from 2014-05-28 06:17:00
v4l2loopback device screen capture displayed in wayland window by krad radio

Web Audio Clone of jkmeter

I’ve long admired the work of Fons Adriaensen.  For some time, I’ve wanted some sort of level indicator for web audio.  So, when I went to implement a meter using the web audio api, I decided I should rip off the wonderful jkmeter for metering jack audio written by Fons.  And the below video shows some results.  What you see in the screencast is the same signal routed to a meter running in chrome, one in jkmeter, and one in firefox.

WebRTC: Improved Krad Radio Signaling and Web UI

Receiving webrtc on firefox for android.
Receiving webrtc on firefox for android.
Brad says goodbye to the Lake-pointed camera at the end of a WebRTC chat session.
The WebRTC interface allows one to move and resize the video streams. In This screenshot, one user has called two different users.
I chat with the cam pointed out the window.


A couple of days ago, oneman helped me design a signaling API for WebRTC in Krad Radio.  From the design, I implemented some very simple server-side WebRTC user management so that a user can “register” with the KR server which can route calls and answers between registered users to set up peer-to-peer WebRTC connections.  On the client side, I added a list of registered users to the javascript KR web ui.  The improvements to both the KR server and the KR web ui lead to the ability for a user to come to the KR web site, register a username, and initiate peer-to-peer video chat with any number of other registered users.  The screenshot below shows me making two separate calls… both to myself, but you see I have three separate video feeds displayed in the web ui: one is the local video feed that I send out, and the other two are feeds from other users, in this case, both me, that I called using the KR WebRTC signalling.

Screenshot from 2013-11-12 09:11:18

Web Audio: “Hello, World!”

Screenshot from 2013-11-09 10:38:16

“Hello, World!” WebRTC

I’m just starting to play around with WebRTC.  Below is a chat session signalled by some WebRTC signalling stuff I added to krad radio.  The javascript involved in all this is pretty intense.  I want to explore possibly using WebAudio to make it beep on connect or something fun.  Yay!

We love to chat.
We love to chat.


Screenshot from 2013-09-27 12:50:38