Baseball Scoreboard - The Transparent Proxy

    If you'd asked me at the start of this project how long I thought this portion would have taken me, I probably would have said a day or two.  Reality didn't work out that way for me.  Instead, I spent more than a month iterating through failures and then making the modifications to get tinyproxy (the transparent proxy of choice) to do what I needed it to.  But before we get to a recap of what I did, a quick start guide (after the jump).

Quick Start Guide

    This quick start is really easy for this section.  ssh into the raspberry pi and execute the following commands:

    cd
    wget https://github.com/ScratchesTheItch/ScoreboardRaspberryPiCode_Configs/raw/master/tinyproxy/tinyproxy-1.8.3-gcscoreboard-bin.tar.gz
    cd /
    sudo tar -xvzf /home/pi/tinyproxy-1.8.3-gcscoreboard-bin.tar.gz

If you downloaded and unpacked the config tar ball in the last post, you should be good to go after a reboot.  If not, revisit the quick start section here (the config contains the necessary changes to rc.local to ensure tinyproxy starts automatically on boot).  Now onto my fails!

Design Goals

   For this portion of the project, I had one goal.  Copy off the relevant part of the conversation between the scorer and the GameChanger (for further processing) while not disturbing the content of the communications (i.e., don't break the existing data flow).

Project Fails

    One of the reasons this part of the project took so long was that my conception of how to do this was deeply flawed and it took several retries to converge on a workable solution.  In order, here's what I tried:

• Passively capture the conversation using tcpdump, reconstruct using perl
• Passively capture the conversation using tcpdump, reconstruct using tcpflow
• Use DNS to redirect api.gamechanger.io to the local interface and then forward to api.gamechanger.io via iptables; passively capture the redirected conversation using tcpdump, reconstruct using either perl or tcpflow
•  Various varieties of DNS, iptables, tcpflow, and perl
• DNS redirection, transparent proxying using perl

At the end of the day,  while the Raspberry Pi is cool, it is a bit underpowered.  In short, every one of the things I did ran into the same problem, lack of system resources.  When run at the same time as the gc-parser (the subject of the next blog post), packets were getting missed when passively captured.  The perl proxy had similar problems (although, in fairness, some of those problems could also have been due to my unfamiliarity with the vagaries of http proxying).  After considering my big ball of fail for a while, I determined that cobbling together command line tools using a script was not going to cut it.  I was going to have to program.

Proxy Program Selection

    At this point, I faced a choice:  either write my own proxy or modify an existing project.  Knowing that compiled code is not my forte nor is web proxying, I chose the to modify an existing project.  Googling around and sensitive to my resource constraints, I settled on tinyproxy (https://banu.com/tinyproxy/) a "light-weight HTTP/HTTPS proxy daemon for POSIX operating systems.1"  I downloaded the latest stable version (tinyproxy-1.8.3.tar.bz2)  and got to work.

Modifying and Building TinyProxy

    A large portion of my time modifying the tinyproxy code was tied up in just trying to understand the code and what portions were responsible for what.  For my application, I was interested in the data portion of the POST requests going to api.gamechanger.io.  After many tries, I was finally able to locate the correct portion of the code and move this project forward.
    The entire modified source can be found here (the entire tar ball of said source can be downloaded at tinyproxy-1.8.3-gcscoreboard-src.tar.gz).  However, in figuring out what I did, I highly recommend examining the diff files I generated (https://github.com/ScratchesTheItch/ScoreboardRaspberryPiCode_Configs/tree/master/tinyproxy/tinyproxy-1.8.3-gcscoreboard-diffs).  Stepping through those diffs, here's what I did to modify tinyproxy to do what I needed.

• conf.c.diff, conf.h.diff - This portion of the program controls reading and parsing the configuration file.  Added an option to specify an output directory to save the data from all the POSTS we're about to receive.
• child.c.diff - by default, tinyproxy spawns 10 child processes immediately after start.  Any and/or all of these processes could be servicing requests at any given moment.  To ensure the output files don't collide, each child is initialized with a different random seed (child pointer + epoch time).
• reqs.c.diff - when a connection is received, this is the portion of the code is responsible for relaying the POST data to the requesting server (NOTE:  This was the part that was hard to find).  Added code to this portion of the code to open a randomly named file, write the POST data to it, and then close the file out once everything is done.
• other diffs - A variety of header structures were changed to ensure that the appropriate subroutines could access the proper data structures in order to pull all of this off.

Once you've appropriately differ the original source, you'll need to build the compiled binary.  First, ensure gcc is installed (i.e., sudo apt-get install gcc).  Then run ./configure --enable-transparent.  The command-line option here is key, as tinyproxy needs to function as a transparent proxy in order for everything to work correctly (i.e., neither the iPad or GameChanger server will be aware of the proxy in between to avoid transmit and decode errors).  Then run make, make test, and finally make install.

Configuring tinyproxy

   In order to get everything to work correctly, we have to set a few configuration options.  These configuration options are set in /etc/tinyproxy.conf (NOTE:  If you downloaded and installed the binary tarball above, your tinyproxy.conf already has these changes).  First off, since we have multiple interfaces, we need to specify which one tinyproxy should listen on.  Since we are sending all api.gamechanger.io requests to the secondary IP address, this is the one tinyproxy should listen to.  That is set with the following option:

Listen 192.168.110.2

The second option we need to set is the output directory for the POST data (the configuration option we added in conf.c above).  For my setup, that has the following value:

OutputDirectory "/home/pi/gamechanger/POSTS/"

    Lastly, we need to add ourselves to rc.local to ensure we get started at boot time (NOTE:  If you installed the config tarball from the last post, this has already been taken care of).  The following line should be inserted into rc.local to get the job done:

/usr/local/sbin/tinyproxy -c /etc/tinyproxy.conf

Conclusion

    Assuming everything is correct, you should be able to reboot the Raspberry Pi and have a working transparent proxy.  The best test is to fire up the iPad GameChanger app, join the scoreboard wifi network, and sync.  If everything is correct, the sync should occur as normal.  If you look in the /home/pi/gamechanger/POSTS directory, you should see files accumulating, indicating that tinyproxy is storing the POST data to be used by the gc-parser, the subject of our next post.

Up next…the GC-parser