Monday, October 29, 2012

Q: What is it like to have the process of video encoding or transcoding bring you to tears?

In 2005, we built a system to do the first amateur ATSC HDTV broadcasts. We broadcast MIT sporting events and the January "Integration Bee" (with play-by-play and color commentary, etc.) on the MIT cable TV system. (

The plan called for us to build our own live HD switcher with wipes and other transitions, score overlay and ATSC encoder -- a commercial system was several hundred thousand dollars at that point. We had four HDV cameras connected via Firewire to a head-end computer, and used an array of 1U servers with nVidia cards to render an arbitrary scene with OpenGL (using a pipeline through four machines to paint the video from four cameras on the frame), then overlay the score and time, and finally encode with A/52 audio in ATSC-conformant MPEG transport stream and send out in QAM256 over the MIT cable system (

Some of the technical challenges proved especially difficult and I was up all night with them before a game. Getting four Firewire HDV cameras to talk to the same computer was a real pain, since these cameras generally have cheap chipsets and all want to talk on the same broadcast channel. (We got this working, but it was quite brittle since unplugging one camera would freeze the whole Firewire bus for about 1 second. In year 2 we switched to connecting the cameras to small Mini-ITX computers and then running them to the switcher over UDP on Ethernet.) Showing up at 10 a.m. at a volleyball game and having to explain to my colleagues that we STILL didn't quite have working video was painful!

The most challenging part was writing the MPEG systems stream encoder that would be ATSC-complaint -- in other words, writing a program to stitch together Dolby AC-3 audio and MPEG-2 video that would actually PLAY in sync on a real store-bought television without glitches. (The MPEG-2 video elementary stream was compressed by libavcodec, and the audio by liba52, but getting a compliant audio-video multiplex is a different story.)

This was difficult because those TVs do not exactly give you a lot of helpful diagnostic information. If you do it wrong, you see glitches, but these can be very rare (like once every 20 minutes!) and it's not like you get a debugging trace. 

ATSC has a lot of requirements you have to comply with, like you can't send a frame of video more than 0.5 seconds before it will be displayed, and if you break these you will see undefined behavior from the TV.

Getting all those pieces put together, so we could actually WATCH our sports broadcasts on a real HDTV in 2005 via the cable TV system without hiccups, was very satisfying and a great payoff for all our work. After spending umpteen all-nighters doing it and breaking numerous promises to my friends and colleagues to have it running earlier, I am sure I was brought to tears (of exhaustion and/or joy) when it finally worked.

Sunday, October 21, 2012

Q: What are France's most remarkable contributions to the modern world?

How about:
  • Aviation (Montgolfier brothers, Robert brothers)
  • Moving pictures (co-invented by the Lumière brothers)
  • Electromagnetism (many contributions by Coulomb and Ampère)
  • Photography (co-invented by Niépce, also Daguerre)
  • Pedal-driven bicycle (Pierre Michaux, Pierre Lallement, and the Olivier brothers)
  • Inflatable automobile tires (Michelin)
  • Stethoscope (Laennec)
  • Gyroscope (Foucault)
  • Fresnel lens (Fresnel)
  • Calculator, probability, and much mathematics (Pascal, Fermat)
  • Galois theory (Galois)
  • Chaos theory, much mathematics (Poincaré)
  • Wavelets (Mallat, Meyer) and fractals (arguably Mandelbrot)
  • Baudot code and "baud" rate (Baudot)
  • Aqualung/SCUBA (Gagnan, Cousteau)
  • Fourier transform (Fourier)
  • Much music (Ravel, Debussy, Bizet, Saint-Saëns, Berlioz, Stravinsky, arguably Chopin)
  • Much, much art (Monet, Renoir, Cezanne, Seurat, Degas, Gauguin, Caillebotte, Rodin, Pissarro, Signac, arguably van Gogh)
  • More here:
  • Friday, October 5, 2012

    Q: How many unique Tweets can ever be posted?

    At least 2^{4200} (a number with 1,264 decimal digits), using the contents of the tweet alone. Plus there is all the metadata, which probably adds at least a thousand bits.

    See . It turns out that Twitter allows almost 2^31 choices per "character," at least when a tweet is first posted. (They decay over time...)

    Unicode itself is a 20.1-bit system, but Twitter doesn't allow literally all Unicode scalar values. (E.g. it messes with < and >.) On the other hand, Twitter does allow the huge characters above the first 2^20, that is to say not Unicode, but below 2^31. (This is almost 31 bits anyway.)

    Disclaimer: I have not checked this myself since writing that blog post in March 2010.