When it comes to raw performance, especially in terms of system load per packet, nothing beats FreeBSD. This has been true for as long I can remember.
Back in 1996, I wrote the first RAM-cached, single-threaded web server. With no special modification, a Pentium-90 MHz single core was able to handle 100 million hits a day while maintaining full network utilization. The CPU had less than a 1.00 load average, yet it was 90% idle, with 50% of the CPU load generated by interrupts caused by the physical network.
This compared with NCSA httpd, which later became Apache HTTPD. It only handled 1 million hits per day on the same machine and OS. When we tried an identically configured Linux OS, my server software got 2 million hits per day, and HTTPD dropped to less than 100k, with a maxed-out CPU and a load average of 9.5. But the network utilization was less than 50%.
I don't have the numbers on hand, but factoring in today's faster machines, today's performance difference is roughly the same. An interesting sidenote: The test program for this server also invented the DDoS as far I know. I never used it that way, but when I traced the source code for some of the early DDOSes, they got it from the test program.
Even though this is a horrible matrix to use, FreeBSD is rock-solid. The longest uptime ever recorded for an actively used server was on a FreeBSD machine that ran a NetWare server, which was up for 18.6 years before it was taken down when its hardware finally failed. Compare this to the longest uptime for a Linux machine: roughly 6 years.
Lastly, FreeBSD is much more secure than Linux; BSD is made by security fanatics for security fanatics ... not as much as OpenBSD, but close. In my HIPAA-related work (see Stan Hanks' answer to Who are Marcas Neal's current clients?), one of our clients used Linux and had several, serious security problems, but as soon we switched the system to FreeBSD, we've had no such issues.
For all of these reasons, it is a no brainer for NetFlix to pick FreeBSD over Linux. What puzzles me, though, is why this is done so infrequently when the advantages of FreeBSD for serious sites are obvious. Toy sites can use whatever they want -- even Windows -- whereas life-critical work requires the ramped-up security of the BSDs.
