Thursday, December 31, 2009

OSS Secret Weapon Deployed

OSS Secret Weapon Deployed

The open source security community through the auspices of the Open Information Security Foundation (OISF) have released the beta version of the next generation intrusion prevention system, Meerkat. The Suricata Engine will run on multiple platforms, and will change the paradigm in malware detection.

Will Metcalf, the maintainer of the Snort Inline project, today made the following announcement on behalf of Matt Jonkman (project leader) of Emerging Threats:

It's been about three years in the making, but the day has finally come!
We have the first release of the Suricata Engine! The engine is an Open
Source Next Generation Intrusion Detection and Prevention Tool, not
intended to just replace or emulate the existing tools in the industry,
but to bring new ideas and technologies to the field.

The Suricata Engine and the HTP Library are available to use under the

The HTP Library is an HTTP normalizer and parser written by Ivan Ristic
of Mod Security fame
for the OISF. This integrates and provides very
advanced processing of HTTP streams for Suricata. The HTP library is
required by the engine, but may also be used independently in a range of
applications and tools.

This is considered a Beta Release as we are seeking feedback from the
community. This release has many of the major new features we wanted to
add to the industry, but certainly not all. We intend to get this base
engine out and stable, and then continue to add new features. We expect
several new releases in the month of January culminating in a production
quality release shortly thereafter.

The engine and the HTP Library are available here:

Please join the oisf-users mailing list to discuss and share feedback.
The developers will be there ready to help you test.

As this is a first release we don't really have a "what's New" section
because everything is new. But we do have a number of new ideas and new
concepts to Intrusion Detection to note. Some of those are listed below:

Amazing that multi-threading is new to IDS, but it is, and we've got it!

Automatic Protocol Detection
The engine not only has keywords for IP, TCP, UDP and ICMP, but also has
HTTP, TLS, FTP and SMB! A user can now write a rule to detect a match
within an HTTP stream for example regardless of the port the stream
occurs on. This is going to revolutionize malware detection and control.
Detections for more layer 7 protocols are on the way.

Gzip Decompression
The HTP Parser will decode Gzip compressed streams, allowing much more
detailed matching within the engine.

Independent HTP Library
The HTP Parser will be of great use to many other applications such as
proxies, filters, etc. The parser is available as a library also under
GPLv2 for easy integration ito other tools.

Standard Input Methods
You can use NFQueue, IPFRing, and the standard LibPcap to capture
traffic. IPFW support coming shortly.

Unified2 Output
You can use your standard output tools and methods with the new engine,
100% compatible!

Flow Variables
It's possible to capture information out of a stream and save that in a
variable which can then be matched again later.

Fast IP Matching
The engine will automatically take rules that are IP matches only (such
as the RBN and compromised IP lists at Emerging Threats) and put them
into a special fast matching preprocessor.

HTTP Log Module
All HTTP requests can be automatically output into an apache-style log
format file. Very useful for monitoring and logging activity completely
independent of rulesets and matching. Should you need to do so you could
use the engine only as an HTTP logging sniffer.

Coming Very Soon: (Within a few weeks)

Global Flow Variables
The ability to store more information from a stream or match (actual
data, not just setting a bit), and storing that information for a period
of time. This will make comparing values across many streams and time

Graphics Card Acceleration
Using CUDA and OpenCL we will be able to make use of the massive
processing power of even old graphics cards to accelerate your IDS.
Offloading the very computationally intensive functions of the sensor
will greatly enhance performance.

IP Reputation
Hard to summarize in a sentence, but Reputation will allow sensors and
organizations to share intelligence and eliminate many false positives.

Windows Binaries
As soon as we have a reasonably stable body of code.

The list could go on and on. Please take a few minutes to download the
engine and try it out and let us know what you think. We're not
comfortable calling it production ready at the moment until we get your
feedback, and we have a few features to complete. We really need your
feedback and input. We intend to put out a series of small releases in
the two to three weeks to come, and then a production ready major
release shortly thereafter. Phase two of our development plan will then
begin where we go after some major new features such as IP Reputation