Is the open-source technology Zeek, one of the most trusted but underappreciated tools in security? Gregory Bell, CEO at Corelight, discusses.
Think back to the mid-1990s. If you’re old enough, you remember the emergence of Mosaic, the first web browser, which was released in 1993 and precipitated the explosion that came to be known as the “dot com” boom. Internet traffic grew exponentially as it was transformed from a DARPA-funded defence and academic network used by few businesses into the platform that drove e-commerce, global communication and the disruption of many industries.
From then through the 2000s, most enterprises embraced the internet quickly, first for emails, but then for virtually every form of communication and business process.
Security on the internet was more or less an afterthought, since most companies were not facing much in the way of systematic attacks, and their networks were for the most part thin, closed and private. Remember, this was the world before smartphones and BYOD, before cloud, before SaaS, and where bandwidth was scarce and expensive.
Government agencies, labs and universities, however were quite different. They operated in a world where major research universities like UC Berkeley, Stanford, MIT, UCLA, Carnegie Mellon and others were interconnected by high bandwidth backbones.
They also had huge and transient populations of students, faculty, staff and visitors who were always pushing the envelope of network throughput, applications, and data. The population turned over much more quickly: roughly 25% of them were replaced each year! Since there was really no “perimeter” or physical access control at a major research university campus, preventing bad actors from getting into a network was pretty tough, unlike a closed corporate campus with defined entry points and closely controlled, stable populations.
The national lab complex, sites like Lawrence Berkeley Lab, Sandia, Los Alamos, Oak Ridge and others were interconnected by an ultra-high bandwidth network that came to be known as ESNet, the Energy Sciences Network.
Since those labs did both highly advanced and fundamental scientific research (e.g. LBL) and nuclear weapons research (e.g. Sandia), they were always ripe targets for attackers. While the weapons labs were highly controlled, the scientific research labs had visiting scientists, grad students and others who came and went.
The Creation of Bro
In this national lab environment, a graduate student named Vern Paxson was doing research on internet traffic growth at the Lawrence Berkeley Lab (LBL) overlooking the San Francisco Bay and the UC Berkeley campus. Paxson had a need to understand what was driving the massive growth of internet traffic so he developed a powerful open source UNIX tool to extract the data elements he was interested in from live network traffic.
He called the project “Bro” after Big Brother in George Orwell’s novel “1984” because with Bro he could see virtually everything on the network, kind of like Big Brother could see everything that Winston was doing.
Bro was quickly recognized as a powerful way to improve security at LBL and was adopted for that purpose. From there it spread quickly to other labs in the Department of Energy (DOE) and then to other agencies in the US government and other research universities like Indiana University, Ohio State, Stanford and many others.
All of these organizations shared those characteristics described earlier: high bandwidth, wide open culture, transient populations and also sophisticated attackers who were after advanced technologies being developed in the US DOE labs and research universities.
Meanwhile, corporate America’s security posture, while it started to get more serious, continued without really being subjected to the sophisticated attacks being faced by the government labs and agencies. So, while Bro was being widely adopted and used in those communities, it was virtually unknown in the corporate world for many years. It also didn’t help that Bro was developed as a UNIX power tool, and not as an enterprise-ready product, so it proved to be quite difficult to implement, integrate and use.
Bro Becomes Zeek
In October 2018 the Bro leadership team decided on a new name after two years of discussion. The word “bro” had come to take on many new meanings in English slang, and in the modern world they decided it no longer fit. The name that was selected - Zeek - alluded to the name of the pseudo client created at LBL when Bro was first created, kind of an inside joke for the team.
Zeek today remains the preeminent network monitoring tool. At its core, Zeek is a real time event processing engine with tremendous flexibility. It supports lots of ways to extend and improve the software, including the deployment of custom packages (scripts) to extract virtually any data required by users.
It can accept threat intelligence feeds from external sources. It can be integrated with third party data via the “input framework” to append data describing facilities, locations, machines or anything else. Zeek researchers are working on new capabilities to develop better insights and detections for encrypted traffic, an area of concern for most enterprises as more and more traffic is encrypted.
Even after 25 years, Zeek remains the preeminent and most capable network monitoring software available. Its scalability, flexibility and adaptability mean it stays relevant as members of the community.
And just like the best open source projects, although undeniably challenging to maintain, Zeek has gathered a thriving community of security professionals exchanging ideas, feedback and advice, in a collective effort to make our networks a little more secure.