Sensitive Data Discovery: Finding Personal Data on File Servers

How do you find files containing national ID numbers, IBANs and card numbers on file servers? Pattern matching, validation algorithms, weeding out false matches and fixing permissions.
Technical summary
Technology
Windows Server file shares, NTFS permissions, PowerShell, Microsoft Purview Information Protection scanner, FSRM
Topic
Technically finding personal and financial data in unstructured data
Level
Intermediate–advanced
Prerequisites
  • A separate service account with read access to the shares to be scanned
  • Basic knowledge of PowerShell and regular expressions
  • An inventory in which data owners are identified
In this article8 sections

Most personal data doesn’t sit in databases but in Excel lists, scanned forms and exported CSV files left on file servers over the years. These files appear neither in the data inventory nor in access reviews. Nobody asks, nobody deletes. This guide explains for system and information security teams how to find data such as national ID numbers, IBANs and card numbers on file servers, how to eliminate false positives with validation algorithms, and how to turn findings into permission fixes and data reduction.

The risk of unstructured data

In structured data (ERP, HR software), access is limited by an application role and logged. On a file server, the same data leaves the application with a single export and is left at the mercy of folder permissions. Situations we run into often:

  • A payroll or staff listing left in a “temporary” shared folder.
  • A customer list exported for a campaign and forgotten.
  • A departed employee’s personal folder left without an owner.
  • Old department folders with read access granted to “Everyone” or “Domain Users”.

In ransomware attacks, these files are the first target for data exfiltration. An attacker copies everything they can access with an ordinary compromised user account; they don’t need an administrator account. That’s why, from an enterprise cybersecurity standpoint, the purpose of discovery doesn’t end with producing an inventory; the real work is shrinking this access area. The full set of technical measures under KVKK is in the implementing KVKK technical measures article; here we focus only on finding the data.

Sensitive data patterns and validation algorithms

If you search with a regular expression alone, the report fills up with false positives: not every 11-digit number is a national ID number, and not every 16-digit number is a card number. Order numbers, serial numbers and phone listings all match. Use pattern matching together with a validation algorithm and, if possible, surrounding keywords (“T.C. Kimlik”, “IBAN”, “Card No”).

Data Pattern Verification
National ID number (TCKN) 11 digits, first digit not 0 Check formula for the 10th and 11th digits
TR IBAN TR + 24 digits, 26 characters in total ISO 13616 mod 97 = 1
Payment card Usually 13–19 digits Luhn algorithm
Phone, email Format pattern No algorithm; by context

The national ID check works like this: the sum of the 2nd, 4th, 6th and 8th digits is subtracted from 7 times the sum of the 1st, 3rd, 5th, 7th and 9th digits; the result modulo 10 should give the 10th digit. The sum of the first 10 digits modulo 10 should also equal the 11th digit. In an IBAN, the first four characters are moved to the end, letters are converted to numbers (A=10 … Z=35), and the remainder of the resulting number divided by 97 must be 1. The PowerShell functions below implement these two checks as an example.

function Test-Tckn([string]$n) {
  if ($n -notmatch '^[1-9]\d{10}$') { return $false }
  $d = $n.ToCharArray() | ForEach-Object { [int][string]$_ }
  $odd  = $d[0] + $d[2] + $d[4] + $d[6] + $d[8]
  $even = $d[1] + $d[3] + $d[5] + $d[7]
  $k10 = ((($odd * 7) - $even) % 10 + 10) % 10
  $k11 = (($d[0..9] | Measure-Object -Sum).Sum) % 10
  return ($d[9] -eq $k10 -and $d[10] -eq $k11)
}

function Test-TrIban([string]$iban) {
  $s = ($iban -replace '\s', '').ToUpper()
  if ($s -notmatch '^TR\d{24}$') { return $false }
  $r = $s.Substring(4) + $s.Substring(0, 4)
  $num = -join ($r.ToCharArray() | ForEach-Object {
    if ($_ -match '[A-Z]') { [int][char]$_ - 55 } else { $_ } })
  return ([System.Numerics.BigInteger]::Parse($num) % 97) -eq 1
}

The algorithm reduces false positives; it doesn’t eliminate them. Some random 11-digit numbers also pass the ID number formula. Use a per-file threshold (for example, at least 10 validated ID numbers) and keyword proximity instead of single matches; only then does the report shrink to a readable size. A thousand-line report doesn’t get read. And a report that isn’t read doesn’t turn into fixes.

Comparing tool options

Tool Strength Limit
PowerShell script No extra cost, quick start Doesn’t read Office and PDF content
FSRM classification Built into Windows Server, schedulable Regular expressions available, no algorithm validation
Purview Information Protection scanner Built-in sensitive information types, labeling Requires SQL Server and an appropriate license
Commercial data discovery products Multiple sources, permission analysis Cost, agent or connector installation

A PowerShell script is a good first step for text-based files (CSV, TXT, exported logs). But know its limit: Select-String doesn’t read the compressed content of DOCX and XLSX files; these formats need a tool that uses IFilter. An Excel file the script calls “clean” may never have been read at all. The example below outputs only the file path and the number of validated matches, without writing the matched values into the report.

$root = '\\fs01\Share'
Get-ChildItem -Path $root -Recurse -File -Include *.csv, *.txt -ErrorAction SilentlyContinue |
  Select-String -Pattern '(?<!\d)[1-9]\d{10}(?!\d)' -AllMatches |
  ForEach-Object {
    $path = $_.Path
    foreach ($m in $_.Matches) {
      if (Test-Tckn $m.Value) { [pscustomobject]@{ File = $path } }
    }
  } |
  Group-Object File | Where-Object Count -ge 10 |
  Select-Object @{n='File'; e={$_.Name}}, @{n='TCKN'; e={$_.Count}} |
  Export-Csv .\tckn_findings.csv -NoTypeInformation -Encoding UTF8

In organizations using Microsoft 365, the Purview scanner provides consistency because it uses the same sensitive information types as DLP and labeling on the cloud side. The built-in types include definitions for the Türkiye national identification number, IBAN and credit card; check the list and confidence levels in your own tenant. In discovery mode, the scanner doesn’t touch files. For installation and configuration, see Microsoft’s scanner deployment documentation.

Scan scope and performance

Scanning produces significant read load and network traffic on the file server. A full scan started during working hours comes back as users complaining “the server is slow”. These decisions are made in planning:

  • Scope: Shared and department shares first, then personal folders. Software distribution and backup shares are out of scope.
  • File types: Office, PDF, CSV and TXT. Video, ISO and executable files are skipped.
  • Scheduling: Outside working hours and at times that don’t clash with the backup window.
  • Incremental scanning: After the first full scan, only changed files are scanned.
  • Service account: A separate, read-only account that can’t sign in interactively. Because it can reach all data during the scan, it’s protected like a privileged account.

Folders the scan can’t access are findings too. A folder the service account can’t read has most likely ended up outside governance, an area with no clear owner. List them separately; don’t throw away “Access denied” lines as errors.

Verifying findings with owners

A scan shows that a file contains values matching a personal data pattern; on its own, it doesn’t show that this really is personal data or that a leak has occurred. The data owner decides why that data is there and whether it will be kept. If you don’t make this distinction clear from the start, the deletion decision falls to IT, and IT shouldn’t make that decision alone. Verification steps:

  1. Group findings by folder owner; if there’s no owner, assign them to the department manager.
  2. Offer three options for each finding: needed and in the right place, needed but in the wrong place, not needed.
  3. Have false positives marked; adjust the pattern or threshold based on this feedback.
  4. Record the decisions and their dates, so the same file isn’t asked about again in the next scan.

Compare the results with the processing activities in the data inventory. If a personal data set not in the inventory was found, either the inventory is incomplete or the data was copied outside its purpose. Both are noteworthy findings. A cybersecurity risk assessment approach can be used to rank findings by risk level.

Fixing permissions and reducing data

In folders containing sensitive data, the first place to look is permissions granted to broad groups. The example below lists broad, non-inherited (explicitly granted) permissions. On Turkish-language operating systems, group names may appear localized; adapt the match list to your environment or check by SID.

Get-ChildItem -Path '\\fs01\Share' -Directory -Recurse -Depth 2 |
  ForEach-Object {
    $folder = $_.FullName
    (Get-Acl -Path $folder).Access |
      Where-Object { -not $_.IsInherited -and
        $_.IdentityReference -match 'Everyone|Authenticated Users|Domain Users' } |
      Select-Object @{n='Folder'; e={$folder}}, IdentityReference, FileSystemRights
  } | Export-Csv .\broad_permissions.csv -NoTypeInformation -Encoding UTF8

The order of remediation doesn’t change. First, who really accesses the data is determined from audit logs or file access records, then a role-based security group is defined instead of the broad group. If you remove the permission without analysis, a business process stops on Monday morning and users start copying the data somewhere else. The problem grows instead of shrinking.

Data reduction gives a more lasting result than fixing permissions. A deleted file can’t be leaked. Files found unnecessary are deleted in line with the retention policy; files whose legal retention period continues but aren’t actively used are moved to an archive with restricted access. Sensitivity labels can be applied to the remaining files, and outbound movement can be controlled with DLP policies. If a move from file servers to the cloud is planned, do discovery before the migration; unnecessary data is then never moved. For the migration steps, see the migrating from file servers to SharePoint guide.

Discovery program checklist

  • Scope, file types and scan scheduling are written down.
  • The service account is read-only and can’t sign in interactively.
  • Patterns work together with a validation algorithm and a threshold.
  • Reports contain no values and are kept in a restricted location.
  • Findings are assigned to owners, and their decisions are recorded.
  • Broad group permissions have been replaced with role-based groups.
  • Scanning is periodic; new findings are tracked.

A one-off scan is a snapshot. Data is produced again every day. To set up discovery together with labeling and DLP as a single program, take a look at the data loss prevention and DLP service page.

Frequently asked questions

Can the scan result be accepted as a complete list?

No. Pattern-based scanning misses PDFs scanned as images, password-protected archives and formats the scanner can’t read; the validation algorithm also accepts some random numbers as valid. The result is a strong indicator for prioritization; the definitive inventory comes from the data owner’s verification.

Should the national ID values found be written into the scan report?

No. The report contains the file path, data type and number of matches. Putting the values in the report copies personal data to a new and often less protected location.

Does the Purview scanner modify files?

When run in discovery mode, it doesn’t apply labels to files and doesn’t change content; it only reports findings. For labeling or protection, the enforce setting has to be turned on. Always run the first scans in discovery mode.

If data moves to SharePoint instead of the file server, is discovery unnecessary?

It’s still needed; it’s more valuable if done before the migration. Unnecessary personal data is deleted or archived before moving to the cloud, and permissions and labeling for the remaining data are designed from the start in the target site.

Support

Contact us for support or a discovery session on this topic.

Contact form
[email protected]

Eretek Cybersecurity

Firewall, VPN, endpoint security and threat prevention. We reduce the attack surface and measure and report the risks.

Knowledge Center

Other articles