Introduction

Intrusion detection systems (IDS) were developed in 1990’s, when the network hackers and worms appeared, initially for the identification and reporting of such attacks. The intrusion detection systems didn’t have the ability to stop such attacks rather than detecting and reporting to the network personnel.

The Intrusion Prevention Systems got both characteristics i.e. threat detection and prevention. The detection process analyzes the events for any possible threats while the intrusion prevention stops the detected possible threats and reports the network administrator.

Purpose & Scope

The main purpose of the project is to evaluate the security capabilities of different types of IDPS technologies in maintaining the network security. It provides detail information about the different classes & components of IDPS technologies, for example, detection methods, security capabilities, prevention capabilities & internals of IDPS. It is mainly focused on different detection techniques & responses by these technologies.

1.2 Audience

The information can be useful for computer network administrators, network security personnel, who have little knowledge about these IDPS technologies.

1.3 Project Structure

The project is organized into the following major structure:

• Section 2 provides a general introduction of IDPS.

• Section 3 provides detail information about of IDPS technologies, components & architecture, detection methodologies, security capabilities & prevention capabilities.

• Section 4 provides the internals of IDPS & incident response.

Section 2: Introduction of IDPS

This Chapter Explains the Intrusion Detection & Prevention Process, Uses, Functions and Different Types of IDPS

The modern computer networks provide fast, reliable and critical information not only to small group of people but also to ever expanding group of users. This need led the development of redundant links, note book computers, wireless networks and many others. On one side, the development of these new technologies increased the importance and value of these access services and on other side they provide more paths to attacks.

During the past, In the presence of firewalls and anti-virus software, organizations suffered huge losses in minutes to their businesses in terms of their confidentiality and availability to the legitimate clients. These modern threats highlighted the need for more advance protection systems. Intrusion detection & prevention systems are designed to protect the systems and networks from any unauthorized access and damage.

An intrusion is an active sequence of related events that deliberately try to cause harm, such as rendering system unusable, accessing unauthorized information or manipulating such information. In computer terminology, Intrusion detection is the process of monitoring the events in a computer network or a host resource and analyzing them for signs of possible incidents, deliberately or incidentally. The primary functions of IDPS are the identification of incident, logging information about them, stopping them & preventing them from causing any damage. The security capabilities of IDPS can be divided into three main categories:

• Detection : Identification of malicious attacks on network & host systems

• Prevention: stopping of attack from executing

• Reaction: Immunization of the system from future attacks.

On the basis of location and type of events they monitor, there are two types IDPS technologies, host-based & network based. The network-based IDPS monitors traffic for particular network segment and analyze the network & application protocol activity for suspicious events. It is commonly deployed at the borders between networks. While on the other hand, host-based IDPS monitors the activity of a single host and events occurring within that host for suspicious activity.

There are two complementary approaches in detecting intrusions, knowledge-based approach and behavior based approach. In knowledge-based approach an IDPS looks for specific traffic patterns called Signatures, which indicates the malicious or suspicious content while in the behavior-based approach an intrusion can be detected by observing a deviation from normal or unexpected behavior of the user or the system.

What is an IDS?

The Intrusion Detection Systems (IDS) can be defined as: tools, methods & resources to identify, assess & report unauthorized or unapproved network activity.

It is the ability to detect attacks against a network or host and sending logs to management console providing the information about malicious attacks on the network and host resources. IDSs fall into two main categories:

• Host-Based Intrusion Detection System (HIDS): A HIDS system require some software that resides on the system and can scan all host resources for activity. It will log any activities it discovers to a secure database and check to see whether the events match any malicious event record listed in the knowledge base.

• Network-Based Intrusion Detection Systems (NIDS): A NIDS system is usually inline on the network and it analyzes network packets looking for attacks. A NIDS receives all packets on a particular network segment via one of several methods, such as taps or port mirroring. It carefully reconstructs the streams of traffic to analyze them for patterns of malicious behavior.

The basic process for IDS is that it passively collects data and preprocesses and classifies them. Statistical analysis can be done to determine whether the information falls outside normal activity, and if so, it is then matched against a knowledge base. If a match is found, an alert is sent. Figure 1-1 outlines this activity.

Response

Manager

• GUI
• Host System
• Pre-processing
• Statistical Analysis
• Alert Manager
• Knowledge
• Base
• Long-Term Storage
• Signature
• Matching

Fig 1.1 Standard IDS System

What is an IPS?

IPS technology has all capabilities of an intrusion detection system and can also attempt to stop possible incidents. IPS technologies can be differentiated from the IDS by one characteristic, the prevention capability. Once a threat is detected, it prevents the threat from succeeding. IPS can be a host-based (HIPS), which work best at protecting applications, or a network-based IPS (NIPS) which sits inline, stops and prevents the attack.

A typical IPS performs the following actions upon the detection of an attack:

• IPS terminates the network connection or user session.

• It blocks access to target .i.e. IP address, user account or sever.

• It reconfigures the devices i.e. firewall, switch or router.

• It replace the malicious portion of an attack to make it benign

An IPS typically consists of four main components:

• Traffic Normalizer: Interpret the network traffic and do packet analysis and packet reassembly & traffic is fed into the detection engine & service scanner.

• Service Scanner: Builds a reference table that classifies the information & helps the traffic shaper manage the flow of the information.

• Detection Engine: Detection engine does pattern matching against the reference table.

Figure 1.2 outlines this process:

• Response
• Manager
• GUI
• Traffic Normalizer
• System Scanner
• Detection Engine
• Alert Manager
• Reference Table
• Long-Term Storage
• Signature
• Matching

FIG 1-2 Standard IPS

Uses of IDPS Technologies

The identification of possible incidents is the main focus of an IDPS, for example, if an intruder has successfully compromised a system by exploiting the vulnerability in the system, the IDPS could report this to the security personnel. Logging of information is another important function of IDPS. This information is vital for security people for further investigation of attack. IDPS has also the ability to identify the violation of security policy of an organization which could be intentionally or unintentionally, for example, an unauthorized access to a host or application.

Identification of reconnaissance activity is one of the major capabilities of IDPS, which is the indication of an imminent attack, for example, scanning of hosts and ports for launching further attacks. In this case, an IDPS can either block the reconnaissance activity or it can alter the configurations of other network devices

Functions of IDPS Technologies

The main difference between different types of IDPS technologies is the type of events they can recognize. Following are some main functions;

• Recording of information regarding observed events, this information could be stored locally or could be sent to the logging server.

• Sending of alerts is one of the vital functions of IDPS. Alerts are sent through different methods i.e. email, SNMP traps, syslog messages etc.

• In case of detection of a new threat, some IDPS do have the ability to change their security profile, for example, when a new threat is detected, it might be able to collect more detail information about the threat.

IDPS not only performs detection but it also performs prevention by stopping the threat to succeed. Following are some prevention capabilities:

• It can stop the attack by terminating either network connection or user session, by blocking access to a target host.

• It could change the configuration of other network devices (firewalls, routers & switches) to block the attack or disrupt it.

• Some IDPS could change the contents of a malicious IP packet, for example, it can replace the header of an IP packet with a new one.

Types of IDPS Technologies

IDPS technologies can be divided into following two major categories:

• Network-Based IDPS

• Host-Based IDPS

Network-Based IDPS

Network-based IDPS monitors network traffic for a particular network segment. They analyze the network and application protocol activity to identify any suspicious activity.

A network based IDPS is usually sits inline on the network and it analyzes network packets looking for attacks. It receives all packets on a particular network segment, including switched networks. It carefully reconstructs the streams of traffic to analyze them for patterns of malicious behavior. They are equipped with facilities to log their activities and report or alarm on questionable events. Main strengths of network-based IDPS are:

• Packet Analysis: Network-based IDPSs perform packet analysis. They examine headers of all IP packets for malicious contents. This helps in detection of the common denial of service (DOS) attack. For example, LAND attack, in which both the source & destination addresses and source & destination ports are same as of the target machine. This cause the target machine to open connection with itself, causing the target machine either performs slowly or crash. It can also investigate the payload of an IP packet for specific commands.

• Real Time Detection & Response: Network-based IDPS detects attacks in real time as they are occurring in the real time and provides faster response. For example, if a hacker initiated a TCP based DoS attack, IDPS can drop the connection by sending a TCP reset.

• Malicious Content Detection: Network-based IDPS remove & replaces suspicious portion of the attack. For example, if an email has infected attachment, an IDPS removes the infected file and permits the clean email.

• Evidence for Prosecution: Network-based IDPS monitors real time traffic and if an attack is detected and captured the hacker cannot remove the evidence. Because the captured attack has data in it but also the information about his or her identification which helps in the prosecution.

Host-Based IDPS

A Host-Based system monitors the characteristics of a single host and the events occurring within that host for suspicious activity. It require some software that resides on the system and monitors the network traffic, syslog, processes, file access & modification and configuration or system changes. It logs any activities it discovers to a secure database and check to see whether the events match any malicious event record listed in the knowledge base. Some of the major strengths of Host-Based IDPS are as under:

• Verification of Attack: Host-based IDPS uses logs which contains events that have actually occurred. It has the advantage of knowing if the attack is successful or not. This type of detection is more accurate and generates fewer false alarms.

• Monitoring of Important Components: Host-Based IDPS monitors key components for example, executables files, specific DDLs and NT registry. All of these can cause damage to the host or network.

• System Specific Activity: Host-based IDPS monitors user and file access activity. It monitors the logoff or login procedure and monitors it on the basis of current policy. It also monitors the file access for example, opening of a non shared file.

• Switched & Encrypted Environments: Host-Based IDPSs provide greater visibility into purely switched environment by residing on as many critical hosts as needed. Encryption is a challenging problem for network-based IDPS but not a major problem for host-based IDPS. If the host in question has log-based analysis the encryption will have no impact on what goes in to the log files.

• Near Real Time Detection: A host-based IDPS relies on the log analysis which is not a true real time analysis. But it can detect & respond as soon as the log is written to and compared to the active attack signatures.

• Real Time Detection & Response: Stack-based IDPS monitors the packets as they transverse the TCP/IP stack. It examines inbound & outbound packets and examines in real time if an attack is being executed. If it detects an attack in real the time then it can responds to that attack in the real time.

Section 2: IDPS Analysis Schemes

IDPSs Perform Analysis: This Chapter is about the Analysis Process- What Analysis does and Different Phases of Analysis.

2.2 Analysis

In the context of intrusion detection & prevention, analysis is the organization of the constituent parts of data and their relationships to identify any anomalous activity of interest. Real time analysis is analysis done on the fly as the data travels the path to the network or host. The fundamental goal of intrusion-detection & prevention analysis is to improve an information system’s security.

This goal can be further broken down:

• Create records of relevant activity for follow-up.

• Determine flaws in the network by detecting specific activities.

• Record unauthorized activity for use in forensics or criminal prosecution of intrusion attacks.

• Act as a deterrent to malicious activity.

• Increase accountability by linking activities of one individual across system.

2.3 Anatomy of Intrusion Analysis

There are many possible analysis schemes but in order to understand them, the intrusion process can be broken down into following four phases:

• Preprocessing

• Analysis

• Response

• Refinement

1. Pre-Processing

Preprocessing is the key function once the data is collected from IDPS sensor. The data is organized in some fashion for classification. The preprocessing helps in determining the format the data are put into, which is usually some canonical format or could be a structured database. Once the data are formatted, they are broken down further into classifications.

These classifications can depend on the analysis schemes being used. For example, if rule-based detection is being used, the classification will involve rules and patterns descriptors. If anomaly detection is used, then statistical profile based on different algorithms in which the user behavior is baseline over the time and any behavior that falls outside of that classification is flagged as an anomaly.

Upon completion of the classification process, the data is concatenated and put into a defined version or detection template of some object by replacing variables with values. These detection templates populate the knowledgebase which are stored in the core analysis engine.

2. Analysis

Once the processing is completed, the analysis stage begins. The data record is compared to the knowledge base, and the data record will either be logged as an intrusion event or it will be dropped. Then the next data record is analyzed. The next phase is response.

3. Response

Once information is logged as an intrusion, a response is initiated. The inline sensor can provide real time prevention through an automated response. Response is specific to the nature of the intrusion or the different analysis schemes used. The response can be set to be automatically performed or it can be done manually after someone has manually analyzed the situation.

4. Refinement

The final phase is the refinement stage. This is where the fine tuning of the system is done, based on the previous usage and detected intrusions. This gives the opportunity to reduce false-positive levels and to have a more accurate security tool.

Analysis Process By Different Detection Methods

The intrusion analysis process is solely depends on the detection method being used. Following is the information regarding the four phases of intrusion analysis by different detection methods:

Analysis Process By Rule-Based Detection

Rule-based detection, also known as signature detection, pattern matching and misuse detection. Rule-based detection uses pattern matching to detect known attack patterns. The four phases of intrusion analysis process applied in rule-based detection system are as under:

• Preprocessing: The data is collected about the intrusions, vulnerabilities and attacks and then it is putted down into classification scheme or pattern descriptors. From the classification scheme a behavior model is built and then into a common format;

• Signature Name: The given name of the signature

• Signature ID: The unique ID for the signature

• Signature Description: The description of the signature & what it does

• Possible False Positive Description: An explanation of any “false positives” that may appear to be an exploit but are actually normal network activity.

• Related Vulnerability Information: This field has any related vulnerability information

The pattern descriptors are typically either content-based signatures, which examine the payload and header of packet, or context-based signatures that evaluate only the packet headers to identify an alert. The pattern descriptors can be atomic (single) or composite (multiple) descriptors. Atomic descriptor requires only one packet to be inspected to identify an alert, while composite descriptor requires multiple packets to be inspected to identify an alert. The pattern descriptors are then put into a knowledge base that contains the criteria for analysis.

• Analysis: The event data are formatted and compared against the knowledge base by using pattern-matching analysis engine. The analysis engine looks for defined patterns that are known as attacks.

• Response: If the event matches the pattern of an attack, the analysis engine sends an alert. If the event is partial match, the next event is examined. Partial matches can only be analyzed with a stateful detector, which has the ability to maintain state, as many IDS systems do. Different responses can be returned depending on the specific event records.

• Refinement: Refinement of pattern-matching analysis comes down to updating signatures, because an IDS is only as good as its signature update.

Analysis Process By Profile-Based Detection (Anomaly Detection)

An anomaly is something that is different from the norm or that cannot be easily classified. Anomaly detection, also referred to as Profile-based detection, creates a profile system that flags any events that strays from a normal pattern and passes this information on to output routines. The analysis process by profile-based detection is as following:

• Preprocessing: The first step in the analysis process is collecting the data in which behavior considered normal on the network is baselined over a period of time. The data are put into a numeric form and then formatted. Then the information is classified into a statistical profile that is based on different algorithms is the knowledge base.

• Analysis: The event data are typically reduced to a profile vector, which is then compared to the knowledge base. The contents of the profile vector are compared to a historical record for that particular user, and any data that fall outside of the baseline of normal activity is labeled as deviation.

• Response: At this point, a response can be triggered either automatically or manually.

• Refinement: The profile vector history is typically deleted after a specific time. In addition, different weighting systems can be used to add more weight to recent behavior than past behaviors.

Section 3: IDPS Technologies

This section provides an overview of different technologies. It covers the major components, architecture, detection methodologies & security capabilities of IDPS.

Components

Following are the major components and architecture of IDPS;

Sensor & Agents: Sensors & Agents monitors and analyze the network traffic for malicious traffic.

Sensor:The technologies that use sensors are network based intrusion detection & prevention systems, wireless based intrusion detection & prevention systems and network behavior analysis systems.

Agents: The term “Agent” is used for Host-Based Intrusion detection & prevention technologies.

• Database Server: The information recorded by the sensors and agents are kept safely in a database server.

• Console: A console is software that provides an interface for the IDPS users. Console software is installed on the administrator’s PC. Consoles are used for configuring, monitoring, updating and analyzing the sensors or agents.

• Management Server: It is a centralized device, receives information from sensors & agents and manages that information. Some management server can also perform analysis on the information provided by sensor & agents, for example correlation of events. Management server can be both appliance based or software based.

3.1 Network architecture

IDPS components are usually connected with each other through organization’s network or through Management network. If they are connected through management network, each agent or sensor has additional interface known as management Interface that connects it to the management network. IDPS cannot pass any traffic between management interface and its network interface for security reasons. The components of an IDPS i.e. consoles and database servers are attached only with the Management network. The main advantage of this type of architecture is to hide its existence from hackers & intruders and ensure it has enough bandwidth to function under DoS attacks

Another way to conceal the information & communication is to create a separate VLAN for its communication with the management. This type of architecture doesn’t provide a much protection as the management network does.

3.2 Security capabilities

IDPS provide different security capabilities. Common security capabilities are information gathering, logging, detection and prevention.

3.2.1 Information gathering

Some IDPS gather general characteristics of a network, for example, information of hosts and network. They identify the hosts, operating system and application they use, from observed activity.

3.2.2 Logging capabilities

When a malicious activity is detected by the IDPS, it performs logging. Logs contain date & time, event type, rating and prevention action if performed. This data is helpful in investigating the incident. Some network-based IDPS captures packet while host-based IDPS records user ID. IDPS technologies allow log to be store locally and send copies of centralized logging server i.e. syslog.

3.2.3 Detection capabilities

The main responsibility of an IDPS is to detect malicious activity. Most IDPS uses combination of detection techniques. The accuracy and types of events they detect greatly depends on the type of IDPS. IDPS gives great results once they are properly tuned. Tuning gives more accuracy, detection and prevention. Following are some the tuning capabilities:

• Thresholds: It is a value that sets the limit for normal and abnormal behavior. For example, the number of maximum login attempts. If the attempts exceed the limit then it is considered to be anomalous.

• Blacklists & Whitelists: A blacklist is list which contains TCP or UDP port numbers, users, applications, files extensions etc that is associated with malicious activity. A whitelist is a list of discrete entities that are known to be benign. Mainly used to reduce false positive.

• Alert Setting: It enables IDPS to suppress alerts if an attacker generates too much alerts in a short time and blocking all future traffic from that host. Suppressing of alerts provide IDPS from being overwhelmed.

3.2.4 Prevention Capabilities

IDPS offers multiple prevention capabilities. The prevention capability can be configured for each type of alert. Depending on the type of IDPS, some IDPS sensors are more intelligent. They have learning & simulation mode which enables them to know when an action should be performed-reducing the risk of blocking benign activity.

3.2.5 Types of Alarms

When IDPS detects an intrusion it generates some types of alarms but no IDPS generates 100% true alarm. An IDPS can generate alarm for legitimate activity and can be failed to alarm when an actual attack occurs. These alarms can be categorized as:

• False Alarms: When an IDPS fails to accurately indicate what is actually happening in the network, it generates false alarms. False alarm fall into two main categories:

• False Positives: These are the most common type of alarms. False positive occurs when an IDPS generates alarm based on normal network activity.

• False Negatives: When an IDPS fails to generate an alarm for intrusion, it is called false negative. It happens when IDPS is programmed to detect ck but the attack went undetected.

2. True Alarms: When an IDPS accurately indicates what is actually happening in the network, it generates true alarms. True alarms fall into two main categories:

• True Positives: When an IDPS detects an intrusion and sends alarm correctly in response to actually detecting the attack in the traffic. True positive is opposite of false negative.

• True Negative: It represents a situation in which an IDPS signature does not send alarm when it is examining normal user traffic. This is the correct behavior.

• ARCHITECTURE DESIGHN

• Architecture design is of vital importance for the proper implementation of an IDPS. The considerations include the following:

• The location of sensors or agents.

• The reliability of the solutions & the measurements to achieve that reliability. For example using of multiple sensors, for monitoring the same activity, as a backup.

• The number & location of other components of IDPS for usability, redundancy and load balancing.

• The systems with which IDPS needs interfacing, including:

• System to which it provides the data i.e. log servers, management softwares.

• System to which it initiates the prevention responses i.e. routers, firewalls or switches.

• The systems used to manage the IDPS components i.e. network management software.

• The protection of IDPS communications on the standard network.

3.3 Maintenance & Operation

Mostly IDPS are operated & maintained by user graphic interface called Console. It allows administrator to configure and update the sensors and servers as well as monitor their status. Console also allows users to monitor and analyze IDPS data and generate reports. Separate accounts could be setup for administrators and users.

Command Line Interface (CLI) is also used by some IDPS products. CLI is used for local administration but it can be used for remote access through encrypted tunnel.

3.3.1 Common Use of Consoles

Many consoles offer drill down facilities for example, if an IDPS generates an alert, it gives more detail information in layers. It also give extensive information to the user i.e. packet captures and related alerts.

Reporting is an important function of console. User can configured the console to send reports at set time. Reports can be transferred or emailed to appropriate user or host. Users can obtain and customized reports according to their needs.

3.3.2 Acquiring & applying updates

There are two types of updates –software updates and signature updates. Software updates for enhancing the performance or functionality and fixing the bugs in IDPS while the signature updates for adding detection capabilities or refining existing capabilities.

Software updates are not limited for any special component but it could include all or one of them i.e. sensor, console, server and agents. Mostly updates are available from the vendor’s web site.

New Chapter

Detection Methodologies

Most IDPS uses multiple detection methodologies for broad & accurate detection of threats but following are primary detection methodologies:

• Signature Based Detection

• Anomaly Based Detection

• Stateful Protocol Analysis

3.3.1 Signature Based Detection

The term Signature refers to the pattern that corresponds to a known threat. In signature based detection, the predefined signatures, stored in a database, are compared with the network traffic for series of bytes or packet sequence known to be malicious, for example, an email with the subject of free screen savers and an attachment of screensavers.exe, which are characteristics of known form of malware Or a telnet