Open Sourcing Castbox

In January 2014 I gave a talk at the Israeli Devcon in Tel-Aviv, named "Chromecast Internals". I announced Castbox at the end of that talk.

Getting that exposure brought up interesting ideas which postponed my plan of open sourcing it, but today, I have no option but to bury these plans due to Google Chromecast changes.

So, much delayed, I'm open sourcing Castbox. The good news is that this project, 8 months later, is more robust (since I wanted to build a business around it).

Castbox still works with most apps and will continue to work until all Chromecast apps migrate to the new Google protocol (which may probably take time).

You can use it to develop your apps and have a Chromecast without the real Chromecast if you want - on a RaspberryPi for example.

Why Go

I have built several other open source projects in Go in the past 2 years, and am running Go in production for a long while. However, I have never stated my opinion and point of view on Go, and I hope to cover some of it below.

My goals for this project were to:

  • Have a build for Raspberry Pi
  • Develop on OSX and run on Linux and Windows
  • Have a reasonably happy development experience
  • Be certain that I will consume low resources and run fast

Flexible Enough

Although it is not really dynamically typed, and is relateively young, Go is flexible. Or in my eyes - flexible enough.

It isn't as flexible as Ruby or Node.js, but in my opinion there's an important realization that to get the same low-level benefit package you'll have to use C, or C++.

With that in mind - I say Go is flexible enough.

Go gives me a rapid development experience, and its ecosystem have exploded to offer good variants of libraries and frameworks in existing ecosystems driven for years by the kinds of Ruby and Node.js communities.

Unlike production services, In this single project I have: a couple of web services, arbitrary JSON handling, arbitrary UDP packets, generated UDP traffic, applied logic, conformed to a noisy protocol, orchestrated processes and real-time communication, and communicated via RPC.

And it felt awesome and refreshing most of the way.

Dependency Management

As with my other projects, I use Gom for managing dependencies and having a fluent development experience.

Since I've started using it a year ago, the problem of dependency management in Go was highlighted in the community and many other dependency managers turned up - yet I found Gom's approach to be very solid and glitch free.

Using this Gomfile:

gom "github.com/codegangsta/martini"
gom "github.com/gorilla/websocket"
gom "github.com/fedesog/webdriver"
gom "code.google.com/p/go.net/websocket"

And gom build closes the deal, without really fiddling with a go home or workspace.

Code once, deploy everywhere

In order to cross-compile (read: build binaries for other OSes and CPU archs on your dev machine) you still need to prepare your Go environment.

For homebrew users - life is happier with the cross-compile-all option.

Using rake, I was able to toggle on GOARCH and GOOS for cross compilation, and wrap everything in a tarball in a nice way.

def go_build(label, arch)
    system({"GOOS" => arch.split('-')[0], 
          "GOARCH" => arch.split('-')[1]}, 
          "go build -o #{label}-#{arch}")
    "#{label}-#{arch}"
end

Where a typical arch looks like windows-386 and linux-arm.

Don't Cares

On a personal note, every community likes its habits and opinions given most, if not all, come from hard-cold justified experience.

I intentionally used my own formatting (against tabs and go-fmt) and conventions which I took from Ruby, my own modeling approach (psuedo-classes extensively) which I took directly off my experience with staticly typed platforms. I think it is reasonable to allow such freedom (and not forcing formatting on a community).

How Chromecast works

It's time to take a quick turn and talk about Chromecast.

When you open the Youtube app on your phone:

  • Youtube signals there's a Chromecast in the air
  • Connect ("Cast")
  • TV switches to Chromecast (CEC)
  • Chromecast plays your video
  • Your phone becomes a rich remote
  • Youtube-specific: multi-users, TV queue

And more technically:

  • Discovery (UDP broadcast based protocol)
  • Chromecast <=> hardware (TV)
  • "Inverted" streaming - don't send resource bytes, just link to resource
  • But not limited to just streaming - any communication is good (games?)

Let's deep dive into some of the interesting things in this project.

Doing some UDP

We'll match the M-SEARCH specified in the DIAL protocol by simple regex matching. We don't need to solve a problem that doesn't exist by building a complex state machine.

(error handling and other cruft cut for brevity)

  msearch := regexp.MustCompile(`(?s)M-SEARCH.*urn:dial-multiscreen-org:service:dial:1.*`)
  socket, err := net.ListenMulticastUDP("udp", nil, udpAddr)
  for {
          message := make([]byte, 4096)
          n, caddr, err := socket.ReadFromUDP(message)
          if !msearch.MatchString(string(message)){
            continue
          }

          /*
            M-SEARCH * HTTP/1.1
            HOST: 239.255.255.250:1900
            MAN: "ssdp:discover"
            MX: 1
            ST: urn:dial-multiscreen-org:service:dial:1
          */

          // build response
          buf := d.config.ExecuteTemplate("dial_response", DialResponse{IP: d.config.Host, UUID: d.config.UUID})
          n, err = socket.WriteToUDP(buf.Bytes(), caddr)
  }

We don't even invest brain cycles for building a response in code. To allow some headroom in maintaining the protocol (in case revisions will change it) I chose to use a template. Here's standard Go template for the UDP response:

var TMPL_DIAL_RESPONSE ="HTTP/1.1 200 OK\r\n"+
                        "LOCATION: http:///ssdp/device-desc.xml\r\n"+
                        "CACHE-CONTROL: max-age=1800\r\n"+
                        "CONFIGID.UPNP.ORG: 7337\r\n"+
                        "BOOTID.UPNP.ORG: 7337\r\n"+
                        "USN: uuid:\r\n"+
                        "ST: urn:dial-multiscreen-org:service:dial:1\r\n"+
                        "\r\n"

The standard Go templating library makes a very maintainable solution.

The communication hub

When building castbox, I discovered that I needed some kind of event hub that can support an offline mode for clients.

That is, a phone can talk to Chromecast when the app is down (Chrome takes time to start and load apps) - and since apps and communication with Chromecast is stateful - the commands must be buffered and replayed against the app when it is started.

Go channels make the perfect solution. Here's that part of a Hub for an App.

(again, cruft removed for brevity)

select {
  case msg, ok := <- app.Channel:
    if !ok {
      return
    }

    hub := msg.To
    from := hub.FromLabel
    to   := hub.ToLabel


    for app.IsRunning && ( len(app.ReceiversHub.Members) == 0  || len(app.RemotesHub.Members) == 0 ){
      log.Printf("%s: Waiting for members to join.", appname)
      time.Sleep(2000*time.Millisecond)
    }


    app.KeepAlive()
    for ws := range hub.Members {
      log.Printf("%s: SEND [%d] %s -> %s (of %d) [%s]", appname, msg.Id, from, to, len(hub.Members), string(msg.Data))
      err := ws.WriteMessage(msg.Mtype, msg.Data)
      if err != nil{
        log.Printf("%s: Error pumping message to %s: %s", app, to, err)
      }
    }
}

and the Hub is simply:

type Hub struct {
  Members map[*websocket.Conn]bool
  FromLabel string
  ToLabel string
  App *App
}

Go Channels make it easy to abstract a pub/sub system, and being that they can buffer events make it ideal for this usecase.

Orchestrating Processes

Being a systems language, Go plays a key role for this use.

The process control facilities were very comfortable to work with and didn't exhibit any glitchiness, I was very happy to not find orphan/zombie processes, redundant processes and the likes over a prolonged usage.

As an example, here's how I start and control Chrome:

  devNull, err := os.Open(os.DevNull) // For read access.
  if err != nil {
    log.Fatal(err)
  }
  attrs := os.ProcAttr{Files: []*os.File{os.Stdin, devNull, devNull}}
  proc, err := os.StartProcess(selectedPath, chromeArgs, &attrs)
  if err != nil{
    log.Fatalf("ChromeDriver: could not start chrome in remote mode")
  }

Summary

There's plenty more to cover, but I chose what I thought to be the most reasonable topics and snippets for a light-read blog post.

There are still web services, more protocols handled, and other ways of orchestrating Chrome. If you want to see some more gory, tricky details, check out the Castbox Github repository.

Feel free to send in pull requests, issues, or drop me a line at dotan@paracode.com :).