642-Yhtomit

Distinguish between alert data (including generation tools) and previously covered NSM monitoring (including collection tools).

250 words
2 sources
APA format
Due Thurs (NO EXCUSES)

Chapter 10: Alert Data: NSM Using Sguil

WHY SGUIL?

Other projects correlate and integrate data from multiple sources. The Automated Incident
Reporting project (http://aircert.sourceforge.net/) has ties to the popular Snort
interface ACID. The Open Source Security Information Management project (http://
www.ossim.net/) offers alert correlation, risk assessment, and identification of anomalous
activity. The Crusoe Correlated Intrusion Detection System (http://crusoecids.
dyndns.org/) collects alerts from honeypots, network IDSs, and firewalls. The
Monitoring, Intrusion Detection, [and] Administration System (http://midasnms.
sourceforge.net/) is another option. With so many other tools available, why implement
Sguil?
These are projects worthy of attention, but they all converge on a common implementation
and worldview. NSM practitioners believe these tools do not present the right
information in the best format. First, let’s discuss the programmatic means by which
nearly all present IDS data. Most modern IDS products display alerts in Web-based interfaces.
These include open source tools like ACID as well as commercial tools like Cisco
Secure IDS and Sourcefire.
The browser is a powerful interface for many applications, but it is not the best way to
present and manipulate information needed to perform dynamic security investigations.
Web browsers do not easily display rapidly changing information without using screen
refreshes or Java plug-ins. This limitation forces Web-based tools to converge on backward-
looking information.2 Rather than being an investigative tool, the IDS interface
becomes an alert management tool.
Consider ACID, the most mature and popular Web-based interface for Snort data. It
tends to present numeric information, such as snapshots showing alert counts over the
last 24 or 72 hours. Typically the most numerous alerts are given top billing. The fact that
an alert appears high in the rankings may have no relationship whatsoever to the severity
of the event. An alert that appears a single time but might be more significant could be
buried at the bottom of ACID’s alert pile simply because it occurred only once. This
backward-looking, count-based method of displaying IDS alert data is partially driven by
the programmatic limitations of Web-based interfaces.
Now that we’ve discussed some of the problems with using Web browsers to investigate
security events, let’s discuss the sort of information typically offered by those tools.
Upon selecting an alert of interest in ACID, usually only the payload of the packet that
triggered the IDS rule is available. The unlucky analyst must judge the severity and
impact of the event based solely on the meager evidence presented by the alert. The analyst
may be able to query for other events involving the source or destination IP
addresses, but she is restricted to alert-based information. The intruder may have taken
dozens or hundreds of other actions that triggered zero IDS rules. Why is this so?
Most IDS products and interfaces aim for “the perfect detection.” They put their effort
toward collecting and correlating information in the hopes of presenting their best guess
that an intrusion has occurred. This is a noble goal, but NSM analysts recognize that perfect
detection can never be achieved. Instead, NSM analysts look for indications and warnings,
which they then investigate by analyzing alert, full content, session, and statistical
data. The source of the initial tip-off, that first hint that “something bad has happened,”
almost does not matter. Once NSM analysts have that initial clue, they swing the full
weight of their analysis tools to bear. For NSM, the alert is only the beginning of the
quest, not the end.

SO WHAT IS SGUIL?
Sguil is the brainchild of its lead developer, Robert “Bamm” Visscher. Bamm is a veteran
of NSM operations at the Air Force Computer Emergency Response Team and Ball Aerospace
& Technologies Corporation, where we both worked. Bamm wrote Sguil to bring
the theories behind NSM to life in a single application. At the time of this writing, Sguil is
written completely in Tcl/Tk. Tcl is the Tool Command Language, an interpreted programming
language suited for rapid application development. Tk is the graphical toolkit
that draws the Sguil interface on an analyst’s screen.3 Tcl/Tk is available for both UNIX
and Windows systems, but most users deploy the Sguil server components on a UNIX
system. The client, which will be demonstrated in this chapter, can be operated on UNIX
or Windows. Sguil screenshots in some parts of the book were taken on a Windows XP
system, and those in this chapter are from a FreeBSD laptop.
I do not explain how to deploy Sguil because the application’s installation method is
constantly being improved. I recommend that you visit http://sguil.sourceforge.net and
download the latest version of the Sguil installation manual, which I maintain at that site.
The document explains how to install the Sguil client and server components step-by-step.
Sguil applies the following tools to the problem of collecting, analyzing, validating,
and escalating NSM information.
• Snort provides alert data. With a minor modification to accommodate Sguil’s need for
alert and packet data, Snort is run in the familiar manner appreciated by thousands of
analysts worldwide.
• Using the keepstats option of Snort’s stream4 preprocessor, Sguil receives TCP-based
session data. In the future this may be replaced or supplemented by Argus, John Curry’s
SANCP (http://sourceforge.net/projects/sancp), or a NetFlow-based alternative.
• A second instance of Snort collects full content data. Because this data consists of libpcap
trace files, Snort could be replaced by Tcpdump or Tethereal (and may have been
so replaced by the time you read this).
• Tcpflow rebuilds full content trace files to present application data.
• P0f profiles traffic to fingerprint operating systems.
• MySQL stores alert and packet data gathered from Snort. PostgreSQL may one day be
supported.
Sguil is a client-server system, with components capable of being run on independent
hosts. Analysts monitoring a high-bandwidth link may put Snort on one platform, the
Sguil database on a second platform, and the Sguil daemon on a third platform. Analysts
connect to the Sguil daemon from their own workstations using a client-server protocol.
Communication privacy is obtained by using the SSL protocol. No one needs to “push” a
window to his or her desktop using the X protocol. Thanks to ActiveState’s free ActiveTcl
distribution, analysts can deploy the Sguil client on a Windows workstation and connect
to the Sguil daemon running on a UNIX system.4 Analysts monitoring a low-bandwidth
link could conceivably consolidate all client and server functions on a single platform.
This chapter explains the Sguil interface and while doing so illuminates the thought
process behind NSM. I start by explaining the interface and use live data collected while
monitoring one of my own networks. I then revisit the case study described in Chapter 4.
Because I used Tcpreplay to relive the intrusion for Sguil’s benefit, the timestamps on the
Sguil events do not match the timestamps on the libpcap traces. I trust this does not
detract from the learning value of the information.
If you would like to try Sguil without implementing all of the server and sensor components,
you are in luck. Curious analysts can download the Sguil client from http://
sguil.sourceforge.net and connect to the Sguil demo server running at bamm.dyndns.org.
Prospective Sguil users can see Sguil in action on Bamm’s server, chat with other users,
and get a feel for the interface before deploying the server components on their own
network.

THE BASIC SGUIL INTERFACE
Sguil relies on Snort for its primary flow of alert data. (If all Sguil did was allow easier
access to Snort alerts, many people would still prefer it to several alternative interfaces.)
Snort alerts populate the RealTime Events tab. (I’ll explain the Escalated Events tab
shortly.) By default Sguil breaks the top half of the screen into three windows (see
Figure 10.1). Alert information is shown in each window, with the top window showing
the most severe alerts, the middle window showing less serious alerts, and the bottom
window showing the least important alerts. These windows correspond to the priority
levels in Snort, with priority levels 1 and 2 at the top, 3 and 4 in the middle, and 5 at the
bottom. Analysts can tweak the sguil.conf configuration file to present a single pane
with all alerts if they so choose. Fonts are also configurable by using Sguil’s File→Change
Font sequence.
The bottom part of the main Sguil display is broken vertically into two halves. The left
side of the screen shows host name and Whois database information, at the discretion of
the analyst. Because DNS queries for host names or lookups for Whois information may
take up to several seconds, many analysts turn these options off unless they need the
information. Sguil does not cache results internally, although the default DNS server usually
will. The bottom of the left side of the screen shows system messages or user messages,
depending on the tab selected. System messages pertain to the amount of space left
on the disk collecting NSM information. User messages appear in an interactive chat
application similar to Internet Relay Chat. Anyone logged in with the Sguil client to the
same Sguil server can communicate via the interface in the User Messages tab. Figure 10.1
shows that user sguil thinks that “Sguil rocks!”
The right side of the bottom of the main Sguil window is dedicated to the highlighted
alert. This varies according to the nature of the alert. Reconnaissance alerts show the sorts
of packets caused by the scan. All other alerts show the packet details in a manner similar
to that used by ACID. Above the packet details you find options for displaying the rule
that generated the Snort alert.
The alert highlighted in Figure 10.1 has a message type of WEB-MISC /~root access.
The ST column on the far left of the top pane shows a value of RT. The ST column refers to
the status of the alert. A status of RT means “real time,” meaning the alert has appeared in
the Sguil interface and is waiting for validation or escalation. This feature hints at the
accountability features built into Sguil. Alerts simply do not scroll off the screen, to be
lost in a database. Analysts must inspect and validate or escalate alerts. (I’ll cover that in
the section Making Decisions with Sguil.) The second column, marked with the CNT
header, shows the count of similar events. Because this WEB-MISC alert has been seen from
the same source IP to the same destination IP 14 times, the CNT field shows that number.
This value increments dynamically while the interface is active.
The third column shows the name of the sensor generating the alert. In this singlesensor
configuration, only the name bourque appears. To the right of the sensor name is
a two-part number representing the sensor and alert number. Here it’s 1.73474, which
corresponds to sensor ID 1, “connection” ID 73474. Beyond the sid.cid field we see a
timestamp, followed by the source IP, source port, destination IP, destination port, and
protocol of the packet or, potentially, the stream that generated the alert. Bringing up the
rear is the alert message.
We see that a packet containing the string /~root headed toward any ports defined in
the $HTTP_PORTS variable (such as 80 TCP) will trigger this alert. If the rule definition is
not sufficient to help the analyst understand the alert, he or she can press the
www.snort.org button, which launches an instance of the defined Web browser. The
URL for the alert will be visited, which in this case is http://www.snort.org/snort-db/
sid.html?sid=1145. On this page the analyst can read Snort’s own documentation for
the WEB-MISC /~root access alert.
If the Show Packet Data button is selected, Sguil shows the packet that triggered the
alert. In our example, it shows the following:
GET /~root HTTP/1.0.
This is the ASCII representation of the application data; the hexadecimal value is also
shown.
On the left-hand side of the screen in Figure 10.1, DNS and Whois information has
been turned on. As a result we see the source IP of 66.92.162.97 resolves to njektd.com,
and the destination IP is a Comcast cable modem. The Whois data for the source IP
shows it belongs to a netblock owned by the Speakeasy DSL ISP.

SGUIL’S ANSWER TO “NOW WHAT?”
At this point you might think Sguil is a cool way to look at Snort alerts. It certainly is, but
we’re only getting started. The question that NSM theory was designed to answer was
stated in the beginning of the book: “Now what?” Now that we have an alert, what does
the analyst do with it? Most commercial and many open source systems leave analysts
with alerts and expect them to make escalation decisions based on the information
present in the alert. The fact that Snort can be tweaked to show the information seen thus
far is a big win for the open source community. Where do we go next?
Sguil is designed to collect alert, session, and full content data. If we have the Snort
sensor configured to log libpcap data for port 80 TCP, we can take the next step using full
content data. If we right-click on the sid.cid field of the highlighted event, we are given
options to query the following items.
• Event History: Show any comments and the validation status assigned by an analyst to
the alert. New alerts marked RT do not have an event history yet.
• Transcript: Generate full content data for the alert, if available. Sguil will query the
sensor for libpcap data associated with the alert, use Secure Copy to transport it to the
analyst workstation, and display the transcript in a new window.
• Transcript (force new): Regenerate the transcript. If the first transcript was created
while the session was still open, a transcript created using force new may show additional
data that was exchanged during the session. Requested transcripts are stored on
the server running the Sguil daemon and used to generate future transcripts for users
who don’t possess a copy of the pcap file on their local workstations.
• Ethereal: Launch Ethereal, reading the same data as would be transferred to generate a
transcript.
• Ethereal (force new): As with forcing a new transcript, this option tells Ethereal to
inspect the latest date for the session designated by the selected alert.
Transcripts are very useful for ASCII-based protocols, like HTTP. For the WEB-MISC
/~root access alert, Figure 10.2 shows part of the transcript.
The “Now what?” question for the WEB-MISC /~root access alert was “Did this attack
succeed?” If the attack succeeded, we might have seen a 200 OK HTTP status code
returned by the target, along with the contents of the /~root directory. Instead we see a
403 Forbidden HTTP status code, indicating the attack did not succeed.
The availability of transcripts is incredibly powerful. While it is tedious to inspect
every alert in this manner, the power of having this sort of data on hand cannot be
denied. There is no ambiguity here because we know as much as the intruder does about
how the victim responded to the attack. After all, we see exactly the same data the
intruder sees. (Of course, encryption obfuscates this form of investigation.)
Certain protocols are not easy for analysts to inspect by using transcripts. Figure 10.1
shows an RPC portmap listing TCP 111 alert at the top of the first pane. This is a good can-
didate for investigation using Ethereal. After highlighting the top alert and right-clicking
on the sid.cid field, we launch Ethereal and see the results shown in Figure 10.3.
Using Ethereal, we see the DUMP Reply tells the intruder what RPC services the target
offers. Again, by looking at the same data as seen by the remote party, we can evaluate
the likelihood of the attack succeeding. Both ASCII and binary full content data help
us understand the nature of the alert and the probability the intruder can accomplish
her goal.
Resolving the alert at hand isn’t the only item of concern. What else has an intruder
attempted? There are two ways to answer this question: queries for alerts and queries
for sessions. By default Sguil supports querying against the source or destination IP
addresses for either form of information. Let’s return to the source of the WEB-MISC
/~root access alert, 66.92.162.97. Right-clicking on the source IP address gives the following
options.
• Query Event Table: The analyst can query for alerts from the source IP, the destination
IP, or from the source IP to the destination IP.
• Query Sessions Table: The analyst can query for sessions from the source IP, the destination
IP, or from the source IP to the destination IP.
• Dshield IP Lookup: The analyst can query on source or destination IP. Querying on
the source IP, for example, sends the URL http://www.dshield.org/
ipinfo.php?ip=66.92.162.97 to the default Web browser. This returns data from the
Dshield database, along with Whois information.
Querying for alerts means asking to see the traffic Snort judged to be suspicious. Querying
for sessions means showing summaries of traffic and letting the analyst decide what
is or is not suspicious. Analyzing session data is potentially more work, but it is a contentneutral
approach. Snort alerts may not trigger on events obscured by encryption or fragmented
by evasion tools. Session data has a greater chance of being recorded for events
that do not trigger Snort rules and thereby lack alert data.
For the first example, we will query for events by right-clicking on the IP address
66.92.162.97 and selecting Query Event Table→Qry Src IP. This action launches the
Query Builder, as shown in Figure 10.4.
Once the Query Builder is started, an analyst can enter SQL statements in the Edit
Where Clause field. By selecting items from the three columns, the Query Builder helps
construct more complicated queries. In most cases, the items requiring modification are
the event.timestamp value (to accommodate queries for older events) or the LIMIT value.
In our example, we leave the defaults and receive the results shown in Figure 10.5.
The screenshot concentrates on the alerts displayed in the main Sguil window. Notice
that the CNT value is 1, so all of the aggregated WEB-MISC /~root access alerts are seen
individually. Besides alerts from the intruder to the target (66.92.162.97 to 68.84.6.72),
Sguil shows alerts triggered by the target’s response. These are ATTACK-RESPONSES 403
Forbidden alerts. Any one of these alerts can be investigated in the same way the original
WEB-MISC /~root access alert was analyzed.
Had we queried for sessions instead of alerts, we would have seen results like those
shown in Figure 10.6. Session data is content-neutral, so Sguil reports any sessions recorded
by the keepstats option of Snort’s stream4 preprocessor. Session results do not appear as
alerts. Certain columns are easy to understand, such as the sensor name, starting and ending
timestamps, and source and destination IPs and ports. The second column, Ssn ID, is a
session identifier. The final four columns provide information on the numbers of packets
sent by the source and destination and on the count of bytes sent by the source and destination.
From the session results window, analysts can generate transcript, launch Ethereal, or
query for any field or combination of fields in the event or session database tables.

MAKING DECISIONS WITH SGUIL
Hopefully by now it’s easy to appreciate the power of investigating events with Sguil. Navigating
through a sea of full content, alert, and session data is not the end game, however.
NSM is about providing actionable intelligence, or interpretations of indications and
warnings, to decision makers. Sguil also helps us manage and classify the events occurring
across our protected domains.
Sguil uses the following alert categories and associated function keys to mark alerts
with those categories in its database.
• F1: Category I: Unauthorized Root/Admin Access
• F2: Category II: Unauthorized User Access
• F3: Category III: Attempted Unauthorized Access
• F4: Category IV: Successful Denial-of-Service Attack
• F5: Category V: Poor Security Practice or Policy Violation
• F6: Category VI: Reconnaissance/Probes/Scans
• F7: Category VII: Virus Infection
• F8: No action necessary
• F9: Escalate
If analysts believe an alert indicates normal activity, they highlight the event and press
the F8 key. If they believe the event indicates an event of categories I through VII, they
mark the appropriate number. If they cannot make a decision, they escalate the alert by
using the F9 key. Note that only alerts can be categorized; session data cannot be classified.
Assume the analyst in our scenario makes a few decisions such that several of the alerts
previously shown have been marked using the appropriate function keys. Once the events
are classified, they are marked in Sguil’s MySQL database with the credentials of the classifying
user and any comments he or she may have made. Aggregated events (i.e., those
with CNT greater than 1) are all marked with the same category if the aggregated event is
highlighted and classified. Figure 10.7 shows an excerpt from the results of the same
query for events to or from 66.92.162.97.

References/Works Sited:

Bejtlich, R. (2004). The Tao of Network Security Monitoring: Beyond Intrusion Detection. Addison-Wesley Professional; 1 edition.