Threat Detection from Firewall Logs: Use Cases

Which threats can be seen in firewall logs? Required fields, normalization, internal scanning, regular outbound connections and data exfiltration scenarios, KQL examples and volume management.
Technical summary
Technology
Syslog, CEF, ECS, SIEM, KQL, Microsoft Sentinel, MITRE ATT&CK
Topic
Building detection rules from firewall traffic and event logs
Level
Intermediate–advanced
Prerequisites
  • Firewall logs being sent to a central SIEM
  • Basic knowledge of a query language (KQL, SPL or SQL)
  • Knowledge of the internal network address plan
In this article8 sections

In most organizations, firewall logs flow into the SIEM, sit there until the retention period expires and are searched retroactively only after an incident. Yet this is the richest network telemetry in the organization. When the right fields are collected and normalized, the same data turns into detections that catch internal network scanning, command-and-control traffic and data exfiltration before the incident grows. The scenarios below are for SOC and security teams who want to build concrete, operable detections from firewall logs in a SIEM.

Which firewall logs are really needed

NGFWs produce multiple log types; their detection value and volume differ:

Log type Detection value Volume
Traffic (allowed) Beaconing, exfiltration, lateral movement Very high
Traffic (blocked) Scanning, misconfiguration, policy violations High
IPS/threat Exploit attempts, known malicious signatures Low to medium
URL/DNS filtering Malicious domains, category violations Medium–high
VPN and identity Account takeover, unusual location Low
System and management Unauthorized configuration changes, HA events Low

Management and VPN logs are low-volume and high-value; collect them in every case. An administrator login to the firewall and a policy change are among the most critical events where an attacker leaves traces. Without these logs, the rest is incomplete. Alongside them, DNS filtering logs complement IP-based firewall logs for domain-based detections.

An often-skipped detail: most NGFWs produce the traffic log when the session ends. A connection that stays open for days may not appear in the log until it closes. If there’s an interim log or session start log option for long sessions, turn it on in critical zones.

Log fields and normalization

Logs from different vendors carry the same concept under different names: one device says srcip, another src, another source_address. For detection rules to work independently of the vendor, logs should be converted to a common schema. In Microsoft Sentinel, the CommonSecurityLog table where CEF logs land does this job, and on the Elastic side, ECS (Elastic Common Schema). The minimum field set for detection:

  • Timestamp (the device clock should be synchronized with NTP and the time zone known)
  • Source IP/port, destination IP/port, protocol
  • Action (allow, block, drop) and the matching rule name or ID
  • Bytes sent and received, session duration
  • Source and destination zone or interface
  • Username and application name (if NGFW identity and application recognition are on)
  • Pre-NAT and post-NAT addresses

If the pre-NAT address is lost, it becomes impossible to tie an external alarm to an internal device. The rule name, in turn, ties detection directly to remediation: you can’t tighten a rule without knowing which rule allowed this traffic. The same information is also used in firewall rule cleanup work. The first place to look in the console is the device clock; a log from a firewall whose NTP has drifted can’t be correlated with other sources. Central collection and forwarding of logs is described in the central syslog server guide on the network site.

Detection scenario: scanning and lateral movement from the internal network

A compromised workstation usually tries to discover targets around it (MITRE ATT&CK T1046, Network Service Discovery) and then spreads over SMB, RDP or WinRM (T1021, Remote Services). The firewall sees this traffic only if there’s a control point between zones. No segmentation, no log.

The logic: connections from a single internal source to many different destinations or ports within a short time window, most of them blocked. In the KQL example below, the thresholds are example values:

CommonSecurityLog
| where TimeGenerated > ago(1h)
| where ipv4_is_private(SourceIP) and ipv4_is_private(DestinationIP)
| summarize Targets = dcount(DestinationIP),
            Ports = dcount(DestinationPort),
            Blocked = countif(DeviceAction in~ ("deny", "drop", "block"))
          by SourceIP, bin(TimeGenerated, 10m)
| where Targets > 50 or Ports > 100

Tuning notes: vulnerability scanners, monitoring servers and network discovery tools trigger this rule constantly. Allowlist these sources but review the list regularly; an attacker using the scanner server is also a real scenario. An additional rule for lateral movement: successful connections from the workstation zone to other workstations on ports 445, 3389 or 5985/5986. There’s rarely a legitimate reason for two workstations to talk over SMB. In environments where a microsegmentation policy has been applied, these flows appearing in block logs provides both detection and control validation.

Detection scenario: beaconing and C2 traffic

Malware connects to its command-and-control server at regular intervals and waits for instructions (T1071, Application Layer Protocol). This regularity is clearly different from human-driven web traffic. Detection logic: for the same source-destination pair, the mean and standard deviation of the time between connections are calculated; pairs whose deviation is very small relative to the mean and whose connection count is high are flagged as candidates.

CommonSecurityLog
| where TimeGenerated > ago(24h)
| where ipv4_is_private(SourceIP) and not(ipv4_is_private(DestinationIP))
| project TimeGenerated, SourceIP, DestinationIP, DestinationPort
| sort by SourceIP asc, DestinationIP asc, DestinationPort asc, TimeGenerated asc
| extend Prev = prev(TimeGenerated), PrevSource = prev(SourceIP),
         PrevDest = prev(DestinationIP), PrevPort = prev(DestinationPort)
| where SourceIP == PrevSource and DestinationIP == PrevDest and DestinationPort == PrevPort
| extend Interval = datetime_diff('second', TimeGenerated, Prev)
| summarize Connections = count(), Avg = avg(Interval), StdDev = stdev(Interval)
          by SourceIP, DestinationIP, DestinationPort
| where Connections > 50 and StdDev < Avg * 0.1

Attacker tools can add randomness (jitter) to the intervals; relying only on standard deviation isn’t enough. A stronger signal is the rarity of the destination: a newly registered or unknown-reputation destination that only one or two devices in the organization connect to regularly. Enrich the result table with how many different internal sources connect to each destination. Common destinations such as update services and monitoring agents are filtered out automatically this way. The rare destination is the real clue.

Detection scenario: unusual outbound traffic and data exfiltration

Data exfiltration usually shows up not through a single indicator but through a combination of several signals (T1048, Exfiltration Over Alternative Protocol; T1041, Exfiltration Over C2 Channel):

  • The amount of bytes a device sends out rising far above its own historical average; especially sessions where more is sent than received.
  • A connection from the server zone to the internet, to a destination normally never seen.
  • Large uploads to the file sharing or cloud storage category outside business hours.
  • Unexpected port and protocol use: non-DNS traffic on port 53, outbound 53 or 853 (DoT) connections from devices other than the internal DNS servers, a non-TLS protocol carried over 443.
  • Geographic anomaly: a user connecting to the VPN from a country never seen before, or egress to regions the organization has no business relationship with.

In volume-based rules, use a per-device baseline instead of a fixed threshold; a volume that’s normal for a backup server is unusual for an accounting workstation. In Sentinel, the SentBytes and ReceivedBytes fields can be used for this calculation. Whether the fields are populated depends on the vendor’s CEF mapping; open a few lines of logs and check before writing the rule. You don’t write a rule on an empty field.

Balancing log volume, filtering and retention

Allowed traffic logs can make up most of the SIEM license and storage cost. To reduce volume without losing detection value:

  • Aggregate high-volume, low-risk flows (from the internal DNS server to external resolvers, backup traffic) at the source or route them to a separate, cheap storage tier.
  • Enable settings that produce multiple logs for the same session (session start and end) only in the zones that need them.
  • Keep the fields used in analytic rules in the hot tier and raw logs in the cold tier.
  • Align retention periods with regulations and internal policy; this design is the subject of the central log retention architecture article.

Tuning and owning detections

In enterprise cybersecurity monitoring, a detection rule is operable only when it has an owner, a tuning history and a response step. For each rule, record: the purpose and ATT&CK mapping, the data source and required fields, the justification for the threshold and allowlist, the expected types of false positives, the checks to do in the first 15 minutes when an alarm comes in, and the last review date. A rule without an owner is noise.

Run new rules in record-only (no-alarm) mode for a few weeks first, review the results and tune the thresholds to the organization’s data. Test rules that never trigger too: run a controlled port scan in an authorized scanning window and see that the scanning rule really raises an alarm. An alarm that never goes off doesn’t mean it works. As a technical reference, you can use the detection recommendations on the MITRE ATT&CK technique pages.

To connect log sources to the SIEM and operate the rule set, the SIEM setup and security logs and SOC and managed security services can be considered together.

Frequently asked questions

Is it really necessary to keep logs of allowed traffic?

From a detection standpoint, yes. Most scenarios such as beaconing, data exfiltration and lateral movement on the internal network show up in allowed traffic; if only blocked traffic is logged, successful attack steps go unrecorded. Manage the volume problem with filtering by log type and storage tiers.

What’s the most common source of false positives in beaconing detection?

Software update services, monitoring agents, time synchronization and the regular check-in connections of cloud applications look a lot like beaconing. Instead of allowlisting these destinations one by one, look at how common the source-destination pair is; focusing on destinations only a few devices go to gives healthier results.

Are firewall logs still valuable when there’s EDR?

They are. Devices without an EDR agent (printers, IP cameras, OT devices, network devices, guest devices) can only be seen through network data. The network behavior of an endpoint whose EDR agent has been disabled also keeps showing up in firewall logs.

How should threshold values in a detection rule be set?

From the organization’s own data. Run the query in monitoring mode over a few weeks of logs, look at the distribution of normal behavior and set the threshold at values that fall clearly outside this distribution. The numbers in ready-made rules are only a starting point.

Support

Contact us for support or a discovery session on this topic.

Contact form
[email protected]

Eretek Cybersecurity

Firewall, VPN, endpoint security and threat prevention. We reduce the attack surface and measure and report the risks.

Knowledge Center

Other articles