Baseball Scoreboard - The Parser

    While I'm presenting the parser near the end of this development, the initial work on this portion of the scoreboard occurred more than a year ago at the conclusion of last season when I was still toying with the idea of the project.  After initialization, the parser steps through the following loop:

1. Wait for a file containing POST data,
2. Unzip the gzipped POST data,
3. Parse the resulting JSON to recover the game state (score/inning, etc),
4. Send the appropriate serial command to the scoreboard,
5. Parse the serial response until the ready prompt is received,
6. Move the POST data file to the output directory

Before we step through how each part works, the quick start guide (starts after the jump):

Quick Start Guide

    In order to get up and running, you'll need to ssh into the Raspberry Pi and do the following:

• Install the Device::SerialPort and Compress::Zlib perl modules (The following command should work, if not see http://www.thegeekstuff.com/2008/09/how-to-install-perl-modules-manually-and-using-cpan-command/ for suggestions):

• sudo cpan -i Device::SerialPort Compress:Zlib
• Download install tarballs into your home directory 
• cd
• wget https://github.com/ScratchesTheItch/ScoreboardRaspberryPiCode_Configs/raw/master/gc-parser/gc-parser.tar.gz
• Unpack tarball in your root directory 

• cd /
• sudo tar -xvzf /home/pi/gc-parser.tar.gz

    At this point, if you installed config.tar.gz from the networking post, you should be good to go with a reboot.  If not, revisit the quick-start portion of that post to ensure the correct entry gets made into rc.local.

Design Goals

    For this portion of the project, my primary goals were to decode the messages copied by the proxy (see previous post), strip out the appropriate information about the game's state, and then feed that information to the scoreboard for display.  Secondarily, I also wanted to make sure that I had appropriate hooks in the parser to support additional functionality (for instance, displaying messages to the fans).

Decoding the POST data

    When the GameChanger app sends information to the server, it does so using gzip compressed POST data (in truth, there are some uncompressed messages, but they don't hold the info we care about).  Here's a small sample file to illustrate what the traffic looks like:

--GameChangerBoundary0xaDKfhSDf1kDS
Content-Disposition: file; name="gzjson"; filename="json.gz"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

\x78\x01\xed\x56\xb1\x6e\xdb\x30\x10\xfd\x15\xe3\x9a\x51\x0a\x28
\x8a\xb2\x64\x6d\x96\x6d\x21\x43\xd1\xa1\xe9\x16\x18\x06\x4d\x51
\x36\x51\x99\x54\x45\xca\x4e\x60\xb8\xdf\xde\xa3\x1c\xa4\x71\xda
\xa0\x43\x11\x64\x89\x26\x52\xc7\x7b\xe2\xbd\x7b\xf7\xa0\x23\xc8
\xbd\xd4\xce\x42\x7e\x77\x84\x4e\xda\xbe\x71\x90\x1f\x41\x98\x5e
\x0f\x0b\xeb\x3a\xf5\x5d\x62\x98\x04\x60\x7a\x7f\x0e\x17\x6b\xde
\x34\x7e\x75\x0a\xc0\x2a\xd7\x73\xa7\x8c\xf6\x59\x5b\xb3\x93\xab
\x46\x69\xd9\xb7\x4a\x57\xf2\x1e\xf2\x38\x00\x7e\xe0\x0f\x97\x2f
\x53\x8f\x60\x3d\xe8\xdd\x32\x80\x2d\x6f\x6a\xc8\xa3\x00\x5a\xe5
\xc4\x56\x76\x1e\xe8\x4a\x55\x90\x43\x12\x67\x24\x8a\x2b\xc2\x48
\x2a\x08\xe1\x6b\x5e\x13\xff\x70\x08\xe0\xaa\x93\x98\x04\x6d\xc3
\x1f\x30\x03\xef\xa1\xb4\x56\x7a\x03\xf9\x80\xed\xdc\x25\x0c\x4d
\x49\x4d\x59\x9d\x0a\xc6\xc6\x49\x5d\x4d\xaa\x89\xa8\xc5\xfa\x4f
\x18\x5f\x8f\x30\x9d\xf4\x57\xf0\xd7\xc6\x6b\x61\xb5\xbe\x2a\xc8
\xe9\x09\xa3\x2b\xf7\xd0\xe2\xfa\xcc\x19\xe6\xb7\x9d\x69\x65\xe7
\x94\xaf\xe5\x08\xc8\x15\x77\x72\x83\x69\x60\x0f\xbc\xc5\x38\xdf
\xad\xd5\xa6\x37\x3d\x86\x5d\xd7\xcb\x00\xb4\x3c\xb4\x06\x77\xb0
\xb8\xc1\xb0\x69\xaa\xf3\x8e\x16\xb8\x3b\x70\x3b\xfd\x7d\xbe\xe6
\x8d\x95\xfe\x42\xae\x93\x7c\xb7\xfa\x2b\x21\xe9\xc0\x07\x89\x31
\x79\x25\xb6\x5c\x6f\xe4\xca\xf3\x46\x49\xc4\x42\x92\x84\x94\x7d
\x8b\xc6\x39\x4d\x73\x9a\xf8\x13\x2f\x20\x26\x92\x10\x91\x8d\x2b
\xc4\x88\xd6\x1e\xc2\x4a\xe7\x06\x0e\x3f\xda\xff\x4a\xfb\xb1\xbf
\xda\x72\x71\x96\x3b\xd0\x32\x9e\xcf\xb2\xf1\x22\x2c\x12\x56\x86
\xac\x9c\x44\x61\x31\x49\x58\x98\xc6\x13\x32\x66\xc5\x7c\x56\xa6
\x33\xa4\xb5\xe5\x28\x10\xa1\x5a\xfe\x34\x65\xa6\x19\xb4\x0f\x3b
\xb3\xf7\x7c\x2f\xf1\xcc\x59\xc7\xaf\x2b\x3f\x1a\x3a\xcd\x10\xee
\x85\xf2\x4f\x98\xfd\x4c\xfb\xb6\x35\x1d\xce\x2d\x14\xd3\xdb\x85
\xd7\x94\x53\x3b\x69\x1d\xdf\xb5\xaf\xca\xc2\xca\x1f\x90\xe3\x84
\x3d\x1b\x44\x61\x2a\x2f\x72\x2f\xcd\x00\x7e\xee\x05\xae\x93\xeb
\xe4\x9a\x5c\x47\xb8\xb7\x5b\xfc\xc2\xaa\x92\x56\x74\xaa\x7d\x64
\xe2\xf6\xac\xf6\xcb\x97\x9f\x07\x1f\xc8\x47\x5f\xd0\x36\x78\x30
\xfa\x44\xe9\xc8\x99\xd1\xe2\x1e\x49\x1c\xdd\xa8\x61\x44\x3d\xfa
\x05\x78\x25\xf7\x4a\xc8\xb3\x52\x19\x99\xe3\xf4\xa7\x59\x38\x4b
\x8b\x69\xc8\xe6\x69\x1c\x4e\x0b\x46\xc3\x2c\x43\xbe\xa7\x59\xc9
\xa2\x92\xe2\xe4\x7f\xb8\x96\x37\x89\xb7\x72\xad\xc1\x85\x9e\xd9
\xd6\x60\x54\x4f\xb6\xf5\xb5\x44\x41\xbc\xa7\x6d\xf9\x81\xf8\xb0
\xad\x7f\xf5\xff\xdd\x6c\x4b\x26\xde\xb6\x22\x81\x5d\x7a\x23\xdb
\xca\xde\xcc\xb6\x8a\xbe\xd3\x16\x6d\x2b\x1a\x7b\xdb\xba\x95\xc2
\xe8\x6a\x54\xe0\x6f\x0b\x16\xf3\xdf\xae\xb5\x3c\xfd\x02\xc6\x8b
\xd9\x2f\x0d\x0a
--GameChangerBoundary0xaDKfhSDf1kDS--

In theory, this should be straight-forward.  Strip off the header and footer (I used the \x0d\x0a\0x0d\x0a\x78\x01 sequence to id the beginning of the gzip content and \x0d\x0a\x2d\x2d sequence to id the end), run it through the Compress:Zlib module, and voila! plain text content.  In reality, there was a little wrinkle that hung me up for the better part of a day trying to troubleshoot.  The problem turned out being that despite my best efforts, perl continued to treat my binary file as a text file, randomly stripping out what it considered to be line ends (hex \x0a).  By inserting a couple of simple checks to restore dropped line ends, the gzip file was made whole and reliable decompression was achieved.  Now that we have uncompressed content, it's time to pull the information we care about.

Parsing JSON

    Once decompressed, we are confronted with a JSON object.  Here's a small sample of that traffic, formatted for readability (not the same sample as given above, for those following along at home, game/player IDs altered as well):

{
   "stream" : {
      "game_id" : "0123456789abba9876543210",
      "shl" : {
…
      "state" : {
         "count" : {
            "outs" : 1,
            "strikes" : 0,
            "balls" : 1
         },
         "situation" : {
            "inning" : 6,
            "half" : 0,
            "bases" : [
               {
                  "base" : 1,
                  "player" : {
                     "$ref" : "player",
                     "$id" : "0123456789abba9876543210"
                  }

                 }

               }

            ],

            "batter" : {

               "$ref" : "player",

               "$id" : "0123456789abba9876543210"

            },

            "home_lineupindex" : 0,

            "pitcher" : {

               "$ref" : "player",

               "$id" : "0123456789abba9876543210"

            },

            "away_lineupindex" : 9

         },

         "score" : {

            "away" : 1,

            "home" : 9

         }

      },

…

Looking at the underlined content above, you can see that the game is 1-9 at the top(0) of the 6th.  Ball-Strike-Out count is 1-0-1 and there is a runner on 1st.  Using regular expressions, the parser pulls those pieces of information and places them in variables.  If nothing has changed since the last update, nothing is done.  If a change occurs, then the change is sent to the scoreboard.

Updating the Scoreboard

    Now that we have a new game state, we have to send the command to the scoreboard for display.  As we covered serial communications to the scoreboard in a previous post, I'll refer you there for more details.  For updating status, the primary command used is SGS (Set Game State).  Given the example above, the command sent to the scoreboard would be:

        SGS 1,9,0,6,1,0,0,1,0,1
    
Once the command is sent, the parser reads out response from the serial bus until hits the ready prompt (Enter command:), at which time, the scoreboard is considered set.

Other Parser Functions

 
     As the gatekeeper to the scoreboard, the parser also implements some additional functionality.  At startup, the parser initializes the USB serial connection and makes sure it's socket options are set correctly (believe it or not, this was a problem for me as most of my original scoreboard programming was done on a Mac).  After initialized, the scoreboard will also convert and send the team logo to the scoreboard (logo is located in the file /home/pi/gamechanger/team_settings/logo.txt).  If the scoreboard detects which side your team is on (using the team id within the /home/pi/gamechanger/team_settings/team_id.txt file), it will then set your team's side (resulting in your logo being shown on that side).  Lastly, the program periodically checks for the existence of a command file (located at /home/pi/gamechanger/command.txt) and, if detected, sends the command to the scoreboard.

Conclusion

   If you have slogged through all of the blog posts and gotten to this point, you should now actually be able to recreate my demo from the start (all of the parts exist for basic functionality).  Have fun!  The next post will cover a little of the gravy that I created to handle routine tasks I didn't want to have to SSH into the Pi for.

Up next …. the Webserver