Central Log Retention Architecture: Which Logs, How Long, Where?

Which records should be kept, for how long and where? Source priority, NTP and integrity, hot/cold retention tiers, deciding retention periods and detecting a stalled log flow.
Technical summary
Technology
Syslog (RFC 5424/5425), rsyslog, Windows Event Log, NTP, SIEM, WORM storage
Topic
Log retention architecture for compliance and incident investigation
Level
Intermediate–advanced
Prerequisites
  • Basic knowledge of Windows audit policy and syslog
  • An inventory of existing log sources
In this article8 sections

You sit down for an incident investigation, look at the domain controller logs, and the oldest entry is four days old. The attacker got in three weeks ago. In most organizations, central log management starts with a SIEM license and a “let’s send everything” approach; what’s left in the end is cost, noise and still incomplete coverage. A log architecture that really serves compliance and incident investigation gives clear answers to four questions: Which sources are collected, how long are they kept, how is it proven that logs haven’t been altered, and who can access the logs?

The two separate purposes of log retention

Logs have two customers, and the design has to serve both. For the security team, logs are for detection (real-time correlation) and incident investigation (retrospective search). On the compliance side, a log is evidence that a control is working: controls 8.15 (Logging), 8.16 (Monitoring activities) and 8.17 (Clock synchronization) in ISO/IEC 27001:2022 Annex A, the logging requirements in the VDA ISA catalog that TISAX is based on, traceability of access to personal data under KVKK (Türkiye’s personal data protection law), and the incident handling area in NIS2 all ask for this evidence.

The two purposes create two different access patterns. For detection, data from recent days must be queryable within seconds. For compliance and forensic investigation, data from months ago must be retrievable, even if more slowly, with its integrity proven. Most architectural decisions follow from this distinction.

Critical log sources and a priority table

The question to ask when choosing sources is: “Which incident would hurt me most if I couldn’t investigate it?” The order below is a reasonable starting point for most enterprise environments.

Source Purpose Priority
Domain controller security logs Authentication, privilege and group changes 1
Entra ID / M365 audit and sign-in logs Cloud identity, mailbox access 1
Firewall and VPN Outbound connections, remote access sessions 1
EDR telemetry and alerts Process, file and malware events 1
Critical server operating system logs Local sign-ins, service and process changes 2
Privileged access and backup system Administrator sessions, deleting and modifying backups 2
Database and business application audit logs Access to sensitive data, bulk exports 2
DNS and proxy C2 communication, traces of data exfiltration 2
Network switches and wireless controllers Configuration changes, port events 3

On Windows, adding the source isn’t enough. If the audit policy doesn’t generate the right events, the logs you collect are empty; that’s the mistake we see often. Verify that these events are really generated: 4624/4625 (successful/failed sign-in), 4672 (privileged sign-in), 4720 (account creation), 4728, 4732 and 4756 (members added to groups), 4719 (audit policy change) and 1102 (security log cleared). For process creation (4688), recording the command line is turned on with a separate Group Policy setting; if it’s off, 4688 gives you only the process name. For syslog forwarding and basic retention settings on network devices, see the setting up a central syslog server for network devices guide, and for building detection rules from firewall logs, the firewall log analysis guide.

The local log size on the endpoint also counts. If the collection agent can’t reach the center for a few hours, events should wait in the local log. The default size fills up quickly on a busy domain controller and overwrites the oldest entries.

REM View the current security log settings
wevtutil gl Security
REM Set the maximum size to 1 GB (in bytes)
wevtutil sl Security /ms:1073741824

Manage this setting permanently with Group Policy (Event Log settings). The one-off command is only for testing; it may be overwritten at the next policy refresh.

Time synchronization and integrity

If you can’t line up events from different sources on a single timeline, your logs lose their evidential value in incident investigation. The design sets up a single time hierarchy: in an Active Directory environment, the domain controller holding the PDC emulator role is fed from reliable external NTP sources, other Windows systems follow the domain hierarchy, and Linux servers and network devices are pointed to the same internal NTP sources. Store logs in UTC or with time zone information; that avoids daylight saving and regional offset confusion. If hypervisor time sync conflicts with the domain hierarchy on virtual machines, the time source is the first place to look in the console.

REM Windows: time source and last synchronization
w32tm /query /source
w32tm /query /status

# Linux (chrony): offset and source status
chronyc tracking
chronyc sources -v

For integrity, three layers are considered together:

  • Integrity in transit: Transport over TLS for syslog (RFC 5425, default port 6514) or the agent’s own encrypted channel. Logs over plain UDP/514 can be altered on the way and silently lost.
  • Integrity in storage: Object lock (Object Lock/WORM) or immutable storage in the cold tier; in addition, generating hashes for daily log files and storing them on a separate system.
  • Administrative separation: The log platform’s administrator accounts should be separate from the administrator accounts of the monitored systems. If an attacker who has taken domain admin rights can’t delete the log platform, the architecture has done its job.

An example configuration for sending from a Linux server running rsyslog over TLS, queuing to disk when the connection drops:

global(DefaultNetstreamDriver="gtls"
       DefaultNetstreamDriverCAFile="/etc/rsyslog.d/ca.pem")

action(type="omfwd" target="log01.example.local" port="6514" protocol="tcp"
       StreamDriver="gtls" StreamDriverMode="1"
       StreamDriverAuthMode="x509/name"
       StreamDriverPermittedPeers="log01.example.local"
       queue.type="LinkedList" queue.filename="fwd_log01"
       queue.maxDiskSpace="1g" queue.saveOnShutdown="on"
       action.resumeRetryCount="-1")

The disk queue prevents log loss while the collector is under maintenance. Calculate the queue size from the source’s log production rate and the outage duration you can accept. By measuring, not guessing.

Retention tiers: hot, warm, cold

Keeping all logs on high-performance SIEM storage for a long time is burning money. A tiered approach uses different storage depending on the age of the data:

Layer Typical content Access Storage
Hot All logs from recent days/weeks Instant queries, correlation SIEM indexes, fast disk
Warm Logs from the last few months Search within minutes A cheaper index or archive tier
Cold Raw logs kept long term Requires a restore Compressed object storage, WORM

Moving between tiers should be automatic, and retrieving data from the cold tier should be tested at regular intervals. An archive that has never been restored is as uncertain as a backup that has never been restored. The immutability options for the cold tier are the same technologies as in the immutable backup options guide.

Deciding the retention period

There is no single “right” period. It’s set by the longest of three inputs and may vary by source type:

  1. Legislation and standards: For example, PCI DSS v4.0 requirement 10.5.1 requires audit log history to be retained for at least 12 months, with the last three months immediately available for analysis. In Türkiye, clarify with your legal department in what capacity and for how long legislation such as Law No. 5651 and its secondary regulations binds you.
  2. Contracts: Customer contracts and assessments such as TISAX may specify a period. Add the log and incident notification clauses in contracts to the inventory.
  3. Incident investigation needs: Weeks, sometimes months, pass between an attacker’s initial access and detection. The retention period for identity, remote access and EDR logs should be long enough to make this retrospective investigation possible.

There’s also a limit in the other direction: logs containing personal data shouldn’t be kept longer than needed for the purpose. That’s why you should keep the retention period per source, with its justification, in a written table, and destroy expired data automatically. How personal data access logs are handled under KVKK is explained in the KVKK technical measures guide.

Controlling log access and privacy

Logs contain usernames, IP addresses, URLs, email subjects, and sometimes passwords accidentally typed into the username field. Apply these principles in access design:

  • Role-based access: the SOC analyst, system administrator and auditor see different data sets and have different rights (read, export, delete).
  • The right to delete and to change retention settings exists only on separate administrator accounts with MFA.
  • Query and export operations on the log platform are logged too; there should be an answer to “who reviewed the logs?”
  • Check during development that passwords, session tokens or full card numbers aren’t written to application logs; if they are, mask them at the source.

Monitoring the continuity of log collection

The most common compliance finding is this: logs are thought to be “collected”, while in fact they haven’t been arriving for months. An agent update, an expired certificate, a changed firewall rule or a full disk silently stops collection. Nobody notices. That’s why you should monitor the expected last event time for each source:

  • Keep the source inventory as a reference list on the log platform and define the expected silence threshold for each source (for example, 15 minutes for a domain controller, 24 hours for a low-traffic switch).
  • Have an automatic alert generated for a source that exceeds the threshold, and have that alert turn into an incident record.
  • A monthly report: the number of sources in the inventory, the number of sources that sent data in the last 30 days, and outage durations. This report is the strongest evidence of coverage in an audit.
  • Define 1102 (log cleared) and audit policy change events as high-priority alerts.

If you need to design source selection, correlation rules and retention tiers together, our SIEM deployment and security logs service sizes this architecture for your existing infrastructure. The syslog protocol is defined by RFC 5424 and the TLS transport layer by RFC 5425.

Frequently asked questions

Is a SIEM required for central log management?

No. Retention and integrity requirements can be met with a central syslog/log collection platform and an immutable archive. A SIEM is needed for correlation and detection. In many organizations, the right architecture is to keep all logs in a cheap archive and the subset needed for detection in the SIEM.

How long should logs be kept?

The period is set per source, based on the longest of the legal, contractual and incident investigation needs. For example, PCI DSS v4.0 requires at least 12 months of retention with the last three months immediately available. Logs containing personal data shouldn’t be kept longer than needed for the purpose.

How do you prove logs haven’t been altered?

Encrypted transport with TLS, WORM or object lock in the cold tier, hash values for log files stored on a separate system, and separating log platform administration from the administrators of the monitored systems are used together. Deletions and setting changes on the log platform should also be logged.

How do I notice that logs aren’t coming from a source?

For each source, an expected last event time and a silence threshold are defined; when the threshold is exceeded, an automatic alert is generated. Once a month, the number of sources in the inventory is compared with the number of sources actually sending data.

Support

Contact us for support or a discovery session on this topic.

Contact form
[email protected]

Eretek Cybersecurity

Firewall, VPN, endpoint security and threat prevention. We reduce the attack surface and measure and report the risks.

Knowledge Center

Other articles