Bill Cheswick's Diary

27 July 2008

Quizling search: Google for "I for one welcome our new * masters".

4 July 2008

Some notes on jailing a web service

I want to run a read-only web service on our home machine. It would be inconvenient if this machine were overrun by malfactors, so I want strong assurance that the web server (and all the other services as well) are unlikely to yield up the machine in the event of corruption. This is precisely the problem I addressed in the Firewalls book, only now it is 2008.

With Ben Laurie's help, I've got some idea of the state of Apache. (These are my conclusions, not necessarily his. Direct your anger at me, if anger there be.

Normally, I would install the latest stable software, as a matter of policy. But apache 2.0 et seq. are a much more complicated rewrite of apache than the original version 1.3. I am looking for simple here, with some of the following considerations:

  • Jailed with relative ease in a chroot area. Ben's belt, my suspenders. The server will run under a user name that lacks write permission in the chroot area. (The except to this may be the log directory, as I used to do, but running the program with fd 2 going to the log is even better.)
  • Speaking of logs, I want apache-formatted logs. This is one of the problems with other solutions like micro_httpd, bozohttpd, and thttpd. These writers pick their own log formats, which I could change. This isn't a show-stopper, but there are lots of tools for analyzing apache logs. I would like to be able to use them. (Come to think of it, this is another avenue of attack: cause the jailed apache to log something evil that corrupts the tools running on it. This security stuff is hard, if you are really paranoid about things.)
  • Read-only access to the web contents. No defacement possible.
  • CGI and other active content handled elsewhere. I need to deal with this, and make it easy. A really good solution would allow safe use of such hacker pits as Perl and especially PHP, the attackers' gateway to Unix machines.
  • Configuration easily rebuilt as new versions become available. Even though I am highly resistant to day-zero problems, why not patch when one needs to.
  • IPv6 would be nice. While doing the work, allow the future to happen.
  • SSL is a must, either though Apache, or stunnel.
  • The version from the ports collection would be nice, since portaudit helps keep me notified of errors. There is a version there that has the bits I want, apache13-modssl+ipv6. Running SSL in the apache server means an Apache corruption releases my private keys. I am not thrilled with that, but faux sites are not a big concern of mine.
  • This is a FreeBSD machine currently running 6.3, though I could upgrade to 7.0 at any time. To compile the released source into a static binary (which is easier to jail):

    $ CFLAGS=-static ./configure
    $ make
    $ file src/httpd 
    src/httpd: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), for FreeBSD 6.3 (603104), statically linked, FreeBSD-style, not stripped

    but it turns out this trick works with the ports collection with the mods I want:

    home:/usr/ports/www/apache13-modssl+ipv6# CFLAGS=-static make
    home:/usr/ports/www/apache13-modssl+ipv6# file work/apache_1.3.41/src/httpd 
    work/apache_1.3.41/src/httpd: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), for FreeBSD 6.3 (603104), statically linked, FreeBSD-style, not stripped

    I am going to have to fiddle with the numerous configure options to layout the chroot sandbox the way I want it. I will be using chroot(8) to jail each call to port 80 or 443, from inetd. So each transfer will have the overhead of httpd initialization, network connection, etc. This lack of memory between connections makes the security much easier to explain (i.e. audit) at the cost of beating the hell out of the CPU/OS. But modern computers have plenty of capacity---50 connections a second should be easy---and all I have to do is tell inetd that the given connection rate is ok. This also scales nicely to multicore CPUs. This will be running on a 2.66GHz Pentium 4 laptop.

    Alas, I am having difficulty configuring httpd properly. I am getting a binary with two builtin modules, and no others. Plus suexec is installed, which I don't need. But configuration is an interplay of FreeBSD ports configuration options, CFLAGS which I am slipping in, and some other configuration options which are processed, but don't seem to effect the final binary. It may just be easier to go to the released code and run with standard configure parameters.

    26 May 2008

    Someone has carved out my particular first appearance on the Hugh Thompson Show here. I have no idea how this site relates to the AT&T tech channel.

    My talk at AUSCERT on Rethinking Passwords went over well with the terrific Aussies. It's a terrific country, and I have the pile of Tim Tams to prove it. Anyway, I think this stump speech is going to have some resonance. It's a stupid problem. If we can't fix this, how are we going to handle some of the really hard problems we are facing?

    13 May 2008

    I recently returned from Greece, my first trip abroad since getting the iPhone. With no open WiFi stations, and extremely expensive data services, my iPhone was, well, just a phone. It removed about 75% of the usefulness of the device.

    I find I count on it now to look up numerous things. I was admonished to look words up when I was in grade school. Now, I do. Except in Greece (and presumably Australia next week.)

    24 April 2008

    isc-dhcpd version 3.0 has not been working on my FreeBSD 6.3 system, and it has been driving me crazy. It generated DHCP responses which the local machine saw with tcpdump, but the packets never made it to the network. I am supposed to know this networking stuff, at least a bit. Why aren't the broadcasts ( working?

    It was tough to come up with the right google query, but finally found a suggestion that lead to the solution. I am running DHCP in a chroot (not a FreeBSD jail) environment. The configuration (make config) in the ports collection allows either promiscuous access to the net, or use of bpf. Mine was set to the former---I don't recall if that is the default or if I thought it would be a good idea to avoid bpf. When I set to use bpf, it Just Works. This was frustrating, and took way more time than it should have. Welcome to the world of networking. And I used to give Lump grief for the need for his CICS foo.

    In the news: I edited my google news pages to include an ego section, and immediately got two hits. The first I already knew about, a fine summary of my Hugh Thompson appearance at RSA among other things. One point is especially important:

    More attacks are going to come in through the browser so it may not matter so much what that the OS underneath is. You go to the wrong page, or the right page that has the wrong advertising agency - you did the right thing on your site but the other guy got hacked. To deal with that there's going to be more sandboxes. I want users to be able to do everything online. I want them to run free in a sandbox. I used ASCII email for twenty years. ASCII email is safe but you want to be able click on the pictures.

    The browser comment comes from a paper I reviewed for the upcoming Usenix Security Conference, a consistently excellent meeting. Serving on program committees is an important way to help keep me up-to-date.

    The paper is based on All Your iFRAMEs Point to Us, which leverages Google's log data with some clever work to measure how much Bad there is out there, and where it comes from. The paper advances the work of another favorite recent paper describing the technology of Strider HoneyMonkey Exploit Detection, more good research from Microsoft.

    The second Google News hit was on a blog entry at the Washington Post by Brian Krebs (have we met?) referring to my true legacy, the simile about crunchy outsides and soft, chewy centers. I am giving myself a retroactive A in High School English for adding a simile to the language. Now, if I can just get them to update my transcript.

    12 April 2008

    I've just returned from Tijuana to pick up a year's supply of Acomplia. Today's price was $129US for a 28-count 20mg supply. (When I did this last June, the price was $125.) I bought 13 of these, which is a year in Sanofi's world. They offer 14 and 28 count packages: check carefully. They take cash or credit, but using credit there may get your card frozen until you can confirm the transaction with the fraud people.

    Customs asked no questions last time. This time, they asked only "What did you buy in Mexico" and I replied "I got a good deal on my diabetes medicine." They waved me through. I was carrying a prescription written by my US doctor in both cases, but wasn't used.

    Rimonabant has improved my blood profile quite nicely, and my doctor is pleased. And the weight loss is nice, but it would be nice if it were more.

    15 Feb 2008

    A friend who is a Treo user asked about my switch to the iPhone. Here's my quick summary:

    I, too, once used a Treo 650, and was generally pleased with it. I had to replace Treo's periodically because the screens wouldn't hold up.

    In most ways, the iPhone is much better. The best are Google maps and the browser, which is a real one, not like the bogus ones on the Treo.

    The fancy keyboard is different, but at least as do-able as the thumb key-ins or graphitti of the past. It takes a little getting used to, but you just have to let it flow. The best times are when you are entering English sentences. The worst are when entering acronyms and initials. It keeps wanting to "correct" them, and I forget to override the corrections.

    What I miss most from the Treo is the memo pad, with its categories. The iPhone has something much simpler and less useful.

    Only one syncing/transition problem: all those photos I had in the contact list on the Treo are gone. Vernor Vinge! I also had a backup problem that lost most of my notes in the memo pad.

    In all, the iPhone has exceeded my expectations, and I am delighted with it. The phone works fine, the interface is a joy, 8GB is plenty for me, and there isn't a single scratch on the front of it, though I have avoided using any protective devices. I run standard Apple software: no jailbreak, no games.

    7 Feb 2008

    It's been a while.

    A new technique, pyrosequencing, has been used to describe an unknown pathogen. It is reported in the current New England Journal of Medicine. Among its other uses, this is a promising technology for understanding What's Going Around, and hence useful for biosecurity surveilance.

    23 Nov 2007

    What's the first indication that one has that a patent has been granted? Certainly, one doesn't hear from the patent office, nor the company that has the patent assigned to it.

    No, one gets a piece of mail from someone like "Patent Awards" of Warsaw, IN, trying to be the first to sell you a plaque made of "the finest solid woods and the highest quality of metal to be found."

    US 7,299,489. My CV needs some cleanup.

    21 Nov 2007

    I love algorithm visualizations. This one by Dan Bernstein was mentioned in a Cryptogram I just caught up on.

    8 Nov 2007 is the talk.

    1 Nov 2007

    My NIST talk should be available soon on google videos.

    5 Oct 2007

    Seem to be stuck around 106.5kg.

    I took a casual walk in a place I remember better than I expected, since it has been 25 years since I have visited the place. From the vault in the Bank of Zork, a hundred zorkmid bill:

    >examine zorkmid
    | 1  0   0         GREAT UNDERGROUND EMPIRE           1  0   0  |
    | 1 0 0 0 0                                           1 0 0 0 0 |
    | 1 0 0 0 0                                           1 0 0 0 0 |
    | 1  0   0                  DIMWIT                    1  0   0  |
    |                      ||||||||||||||||                         |
    |                      ||   __  __   ||              B30332744D |
    |                      ||  -OO  OO-  ||                         |
    |       IN FROBS       \||    >>    ||/       WE TRUST          |
    |                       ||  ______  ||                          |
    | B30332744D             |  ------  |                           |
    |                        \\________//                           |
    | 1  0   0    Series       FLATHEAD     LD Flathead   1  0   0  |
    | 1 0 0 0 0   719GUE                     Treasurer    1 0 0 0 0 |
    | 1 0 0 0 0                                           1 0 0 0 0 |
    | 1  0   0        One Hundred Royal Zorkmids          1  0   0  |

    2 Oct 2007

    It occurred to me today following a discussion at work that an iPhone would make a nice remote control for a MythTV, when the bluetooth support gets better. Then I remembered that WiFi should be just fine. It turns out that Myth has fine hooks.

    It has already been done, and quite nicely. I have some complaints about Myth's design, but this was as easy as it should be. I want to change some of the buttons (easy).

    Lorette is gonna plotz.

    25 Sep 2007

    I picked up a nice small UPS out of the hallway trash at work today. I think one measure of the quality of a research lab might be the quality of its trash.

    24 Sep 2007

    104.2kg. This was surprising, and I checked it twice. True, I had a period of fasting and prep for a medical test, but it doesn't seem like that would make that much difference. Below 100kg before Christmas?

    It's time to build an actual graph, rather than these log entries.

    19 Sep 2007

    105.6kg, a new low. And it feels like I am eating a lot.

    18 Sep 2007

    IPv6 is not so easy to deploy for an ISP. It's going to take some incentive to go through all this, and that incentive is user demand, which I am guessing isn't that much at the moment.

    Well, I am in google news a couple of times. A simple answering machine messege makes the news at Network World! Really, I didn't realize I had said "iPhone" until they pointed it out and really, AT&T had nothing to do with it.

    Of course, I AM delighted with my iPhone---it is a masterpiece---and I give my company, AT&T, top marks for being involved. I presume that PR will like that I say this, but they haven't returned my calls yet.

    8 Sep 2007

    The funny thing about rimonabant is that brownies and fries are OK, now. I simply have much less appetite. If I eat a piece of fudge, it is just one piece, because I am done. If I have the fries, I will eat less at the next meal. This seems to be the case even when I pig out, which is rare: I bought a package of Vienna Fingers. I've learned not to do this in the past, because I will simply eat the whole box. I didn't, it took a couple of days to go through the package. It is now much clearer that I have to think about nuitrition more, since I am really eating only two meals a day, plus an apple or something. (Sugar/carbs cravings are down, and often a piece of fruit will do the trick for a snack, when in the past they didn't hack it. 106.4kg today, down from 112kg in late June.

    5 Sep 2007

    107.0 kg

    1 Sep 2007

    In my judgement, most of the current fears about global warming are seriously misplaced. We don't know anywhere near enough about the patient to diagnose and treat the planet, and in any case, most of the dire scenarios play out over a period of many decades. The assumption is that our economic frog will boil as well, something that reminds me of statists' estimates of the results of budget cuts. These assumptions ignore modern economic and political systems that have dealt with much greater change (automobiles? population growth?) than most dire warming predictions. Beachfront property in northern Canada is cheap right now.

    Actually, I'd be much more concerned about a returning ice age. If there is a big lever to push, you'll find me on the cold side, pushing towards warmer.

    Freeman Dyson addresses global warming, and he lays out the skeptic's argument better than I can. If you have seen Al Gore's screed, but haven't read this, you are probably misinformed.

    BTW, I have had the pleasure to meet Freeman and his wife and some grandkids, and know Esther and especially George quite well.

    While you are in a skeptical frame of mind, check out Bjørn Lomborg's Copenhagen Consensus 2004, which explores current world problems and cost-effective ways of dealing with them. Money for global warming is way down on the list. Alas, Congress loved Al, and mostly ignored Bjørn, when they both testified.

    14 Aug 2007

    106.6 kg

    I was going through some old logs today. Here's a little piece of memory lane:

            Date: Tue, 3 Oct 89 10:42:27 PDT
            From: HOSTMASTER@NIC.DDN.MIL
            Sender: SUE@NIC.DDN.MIL
            cc: HOSTMASTER@NIC.DDN.MIL
            Reply-To: HOSTMASTER@NIC.DDN.MIL
            In-Reply-To: Message from "" of Tue, 3 Oct 89 02:19:37 PDT
            Message-ID: <12531200902.34.SUE@NIC.DDN.MIL>
            I have a few questions before adding your new gateway to the host table.
            First thing I'd like to point out is that we already have a host entry
            called RESEARCH.ATT.COM and it's currently acting as the primary server
            for ATT.COM.
            [PHOTO:  Recording initiated  Tue 3-Oct-89 9:40am]
            @who host
            AT&T Bell Laboratories (ATT)
               Building 2C, Room 501
               600 Mountain Avenue
               Murray Hill, NJ 07974-2070
               Hostname: RESEARCH.ATT.COM
               Nicknames: ATT.ARPA
               System: VAX-11/750 running UNIX
                  Cheswick, Bill  (BC71)  ches@RESEARCH.ATT.COM
                  (201) 582-7389
               domain server
               There are 5 registered users:
            Cheswick, Bill (BC71)           ches@RESEARCH.ATT.COM           (201) 582-7389
            Feigenbaum, Joan (JF10)         jf@RESEARCH.ATT.COM             (201) 582-6910
            Kowalski, Thaddeus J. (TJK9)    FRODO@ATT.ARPA                  (201) 582-2771
            Merritt, Michael (MM320)        mischu@ATT.ARPA                 (201) 582-5334
            Presotto, David (DP58)          PRESOTTO@RESEARCH.ATT.COM       (201) 582-5213
            [PHOTO:  Recording terminated Tue 3-Oct-89 9:41am]
            This means that we cannot add RESEARCH.ATT.COM as an alias for the new
            gateway, INET.ATT.COM, which is what I think you were requesting.  If
            I'm wrong, please tell me so, if not, we'll need to clear up the
            duplicate hostname before acting on your request.

    12 Aug 2007

    107.2 kg

    glassesses: Haven't been using the flip-up portion since the late fall, when I lost the reading part again. Alas, I am told the style is no longer manufactured. I have a couple more flip-ups for reading, but am down to my last frame. How do I get the lenses updated?

    In any case, myopia is a fine treatment for presbyopia. I read a lot of books with my nose up close.

    23 July 2007

    This older paper by Ross Anderson is a fine summary of the economics behind today's computer insecurities. The last paragraph is not a bad summary of the wider security challenges of religeous extremism we face today.

    19 July 2007

    SOUPS conference. Nice game to teach naive users about evil URLs. I learned a little. There are a couple tricky forms they didn't teach, like including the domain name in the URI, which often looks about right. Of course, we are not going to teach the whole world to get this right, but it is not a bad thing to try.

    The &^*&^ mute button on the iphone doesn't stiffle all sound: it will still play ipod music out loud. I learned this to my embarrassment when I tapped the wrong place during a talk. I ended up turning the volume down to zero after attempting to turn it off with several buttons.

    17 July 2007

    So far, my iphone has pretty much exceeded my expectations. But today, I tried connecting to a non-configured VPN, while just messing around. Nothing was happening, which is fine. But the a call came in. No amount of touching and sliding would answer the phone. The caller went away. A few seconds later the phone freed up, and I was able to call back and finish the business.

    Lack of concurrency has always been my biggest beef with PalmOS. Some things just lock it up until finished, especially phone network things with obnoxious timeouts. It's too bad Apple has a little of that in the iphone.

    I met Sean today, and remembered his name. This is exceedingly rare that I remember names upon initial contact. What was different this time? My glasses were off (iphones are fine for presbyopics without their glasses) when Dave and Sean breezed past my room, with a waved invite. I only saw a blur.

    When I actually met (and saw) Sean, I still remembered his name. It would seem that the meet-and-greet routines in my brain short-circuit the name-catching parts. I am going to have to try some new, bizarre, and probably fame-making protocols when meeting people to actually remember their names. Close my eyes? Avert vision? Flip glasses?

    I've ridden twice to work on the electric bike. It's a workout, but works great. Also made a couple tours of the neighborhood. Excellent!

    There appear to be two hazzards on the road: wide trucks, and motorists who are unaware of the status of bikes as actual vehicles on the roads.


    2 July 2007

    Electric bike

    I rode the electric bike into work today, the first time I have ridden to work in 30 years.  I am dubbing it the HDL Local. The weather was perfect, the traffic on this July 4th week was a bit lighter, and it wasn't bad at all. I do worry about the drivers.  One mistake on their part, and I am toast.  One mistake on my part, and I am toast.

    I used a little less than half the battery, a little more of my energy budget than I wanted to.  Of course, I don't mind arriving home a bit more sweaty, so I think it will work out OK until the battery capacity diminishes with age. By that time, I should be in better shape.

    The recumbent part works great for me: no particular aches or pains.  I do have more trouble navigating through traffic and making left turns. This all leaves me plotting alternative routes based on traffic, turns, lights, etc.


    I am at 110 kg this morning.  The travel weight came off quickly, and might have little to do with the rimonabant.  It seems to be free of side effects, with the exception of a curious flutter in my chest when I sleep.  This has to be investigated, of course.  My Dad had atrial fib for many years.  The literature says there are no CB1 receptors in the heart, which is why cannabinoids are used for analgesia in heart patients.

    21 June 2007

    Von Ahn's photo labeling game: The ESP game.

    Rimonabant started. Weight about 112 kilos, up from 109 at the end of my sabbatical. AT&T Research appears to be fattening.

    19 June 2007

    More on the Mac.

    My new electric bike.

    I've got a three-month supply of Rimonabant. I will write this up shortly. I will start on it after this trip to Santa Clara is done.

    5 June 2007

    I've been Mac-positive for over four weeks now. I am not going back. And the stock is being very nice, too.

    My badge arrived today, almost two months after start of work. Got the pro-card yesterday. My first expense report was rejected for about 7 different reasons, including non-use of the pro-card.

    Things have settled down enough that I actually took time to clean up a lot of the exhibit code right here on the Mac. The port has been generally as easy as I expect Linux to be. I haven't figured out the Magic Code to grab video, but I will do that, encase it in my simple API, and move on.

    The emergency holographic keynote for SRUTI is coming along pretty well. I hope there is time for a cookie talk practice session before I head west.

    I wonder how long it takes to buy stuff at AT&T. My real Mac hasn't arrived yet, nor my LCD monitors. I want to play with some of the ad hoc wireless Zigby remotes from Berkeley/Intel, and a decent touch screen for the exhibit stuff. I think an exhibit needs to be in FP somewhere where visitors can place with it.

    Netsci 2007

    22 May 2007

    I am at the Netsci conference in Flushing Meadows, NY. It's quite close to the old 1964-1965 World's, which was excellent and I guess we don't do those any more.

    This conference is similar to, but smaller than, one of the top conferences I have ever attended, Networks: Structure, Dynamics and Function about four years ago. There are people from a number of disciplines, especially biology, discussing graphing, networks, and network visualization.

    There were two talks that especially caught my interest today. Reka Albert of Penn State showed a visualization of the various stages of human immune response to attacks by a couple different bacteria. I had never seen the steps shown so clearly, especially when each kicks in on which day following infection. Additional amazing visualizations included how re-infections are handled, and how antibiotic administration changes the course of the disease, when it does. Her visualizations are on the last two pages of this preprint. The paper also addresses several analyses of the immune system interactions and how various bacteria try to interfere with it.

    Peter Hook showed several visualizations of the relationships of voting records of Supreme Court justices over the past 50 years. You can find his stuff here, under Netsci. By the way, he presented this material without bias. It was refreshing to hear a politically-related discussion without the barking political Tourette's that is so common today.

    20 May 2007

    Coined a word today. "Cooliscentii". Used when referring to Peter Honeyman, who had just clued me into twitter.

    Previous diary