Security Engineering / Tools / Google SecOps SIEM
Tool 03 · SIEM

Google SecOps Formerly Chronicle. Two things make it different from every SIEM you have used: UDM, a strict normalised schema everything is parsed into, and YARA-L 2.0, a rule language built for time-windowed correlation rather than SQL-with-extras. Learn those two and the rest of the platform is navigation.

needs → Detection engineering 5 tiers · 17 modules UDM · UDM Search · YARA-L
Tier 0 · Groundwhat it is

The platform in one page

Module 0.1

Architecture and the design bet

Why searches over a year still return quickly

SecOps ingests security telemetry, parses it into a single schema, indexes it against an entity graph, and runs detection rules continuously over both live and historical data. Retention is typically a year at a flat capacity price rather than per-GB-per-day, which changes behaviour: you stop rationing what you send and stop deleting history to save money.

1Ingest — forwarder, feeds, ingestion API, or direct cloud integrations.
2Parse — vendor log → UDM fields. Raw log is kept alongside.
3Enrich — entity context (asset, user), Google threat intelligence, WHOIS, prevalence, geo.
4Detect — YARA-L rules, curated detection sets, IoC matching.
5Investigate — UDM Search, entity views, timelines; hand off to SOAR for case management.
Module 0.2

Getting data in

Four routes, and the log-type concept
MethodBest forNotes
Forwarder / Bindplane agentSyslog, files, Windows events on-premA container or service you run; needs monitoring itself
FeedsSaaS and cloud APIs (Workspace, AWS, Okta, Netskope)Configured in the console; scheduled pulls
Ingestion APICustom sources, in-house appsPush UDM directly and skip parsing entirely
Native cloud integrationGCP, and direct exportsLowest friction where available

Every source is tagged with a log type (e.g. WINEVTLOG, AWS_CLOUDTRAIL, OKTA, NETSKOPE_ALERT, GMAIL_LOGS). The log type selects the parser, so getting it wrong is the usual reason fields arrive empty.

First thing to verify on any new source: not that data is arriving, but that it is parsed. Search for the log type and inspect a sample event's UDM fields. Data that lands as raw text with an empty principal is invisible to every rule you write. Ingestion dashboards showing green volume are not evidence of usable telemetry.
Tier 1 · Mechanicsthe schema

UDM — the Unified Data Model

Module 1.1

The noun structure

principal, target, src — who did what to what

UDM describes every event with the same grammar. Learn these six top-level objects and you can read any event from any product.

ObjectMeaningTypical contents
metadataAbout the event itselfevent_timestamp, event_type, product_name, vendor_name, log_type, product_event_type
principalThe actorhostname, ip, user.userid, process.command_line, asset
targetWhat was acted uponhostname, ip, port, url, file.full_path, user, registry
srcSource when distinct from principalOften used in network and proxy events
observer / intermediaryThe device that saw or relayed itProxy, firewall, mail gateway
security_resultVerdictaction (ALLOW/BLOCK), severity, rule_name, threat_name, summary
networkNetwork specificsdns.questions.name, http.method, ip_protocol, sent_bytes
extensionsDomain-specific extrasauth.mechanism, vulns

Read it as a sentence: principal did event_type to target, with security_result as the outcome. "User jdoe on host LAPTOP-01 (principal) performed NETWORK_DNS for cdn-metrics.io (target/network), result ALLOW."

Module 1.2

Event types you will use constantly

The vocabulary that makes rules portable
event_typeFires forSourced from
PROCESS_LAUNCHA process startedEDR, Sysmon, 4688
NETWORK_CONNECTIONA connection was madeEDR, firewall, flow logs
NETWORK_DNSA DNS queryDNS servers, EDR, resolvers
NETWORK_HTTPA web requestProxy/SWG (Netskope), WAF
USER_LOGINAuthentication attemptIdP, Workspace, Windows, SSH
USER_CHANGE_PERMISSIONSPrivilege modificationAD, cloud IAM, SaaS admin
FILE_CREATION / FILE_MODIFICATIONFile activityEDR
REGISTRY_CREATION / _MODIFICATIONRegistry writesEDR, Sysmon
EMAIL_TRANSACTIONA message eventGmail, email security
RESOURCE_CREATION / _DELETIONCloud resource lifecycleCloudTrail, GCP audit
SCAN_UNCATEGORIZED, GENERIC_EVENTEverything elseVarious
Why this pays off: one rule keyed on USER_LOGIN with security_result.action = "BLOCK" covers Workspace, Okta, AWS console, SSH and Windows simultaneously. Writing the same detection five times per product is the cost you avoid by learning the schema first.
Module 1.3

Entities and context enrichment

The graph that makes an IP mean something

Alongside events, SecOps maintains entities — assets, users, resources — assembled from context sources (asset inventory, IdP directory, EDR, vulnerability data). This is why clicking a hostname gives you an owner, an OS, recent alerts and prevalence rather than a bare string.

  • Asset context — is this a server, a laptop, whose is it, which site?
  • User context — department, manager, groups, employment status. Enables rules like "privileged group change involving an account marked as departed".
  • Threat intelligence — Google's own feeds and Mandiant intelligence applied to observed indicators.
  • Prevalence — how common is this domain or hash across your environment and globally. Rarity is a first-class signal here rather than something you compute manually.

Entities are referenced in YARA-L via the entity keyword in a graph join, letting a rule ask "was the involved user, at the time of the event, in the finance group?" — context most SIEMs force you to hard-code into lists.

Tier 2 · Operatesearching

UDM Search

Module 2.2

Reference lists

Keeping data out of rule logic

Reference lists are named collections — strings, regex patterns or CIDR ranges — usable in both searches and rules. They are the correct home for anything that changes independently of detection logic.

// in a rule or search
not $e.principal.ip in cidr %corporate_egress_ranges
$e.principal.process.file.full_path in %approved_admin_tools
$e.target.domain in %known_bad_domains

Good candidates: corporate egress and POP ranges, approved admin tooling paths, service-account names, break-glass accounts, executive accounts for high-touch rules, and vendor domains. Bad candidate: anything that encodes the logic of the detection rather than its data — that belongs in the rule so it is visible in review.

Tier 3 · EngineerYARA-L 2.0

Writing detection rules

Module 3.1

Rule anatomy

Five sections, in a fixed order

YARA-L borrows its shape from YARA but operates over time-ordered events rather than file contents. Every rule has the same skeleton:

rule suspicious_encoded_powershell {

  meta:
    author      = "secops"
    description = "Encoded PowerShell launched by an Office application"
    severity    = "HIGH"
    mitre_attack_tactic    = "Execution"
    mitre_attack_technique = "T1059.001"
    reference   = "internal-runbook-042"
    version     = "1.1"

  events:
    $e.metadata.event_type = "PROCESS_LAUNCH"
    $e.principal.process.parent_process.file.full_path = /winword|excel|outlook/ nocase
    $e.principal.process.command_line = /-enc |-e |frombase64string/ nocase
    $host = $e.principal.hostname          // placeholder for grouping

  outcome:
    $risk_score = max(85)
    $user       = array_distinct($e.principal.user.userid)

  condition:
    $e
}
SectionRequiredPurpose
metaRecommendedDocumentation and ATT&CK mapping; surfaces in alerts
eventsYesPredicates the events must satisfy; declares variables and placeholders
matchMulti-event onlyWhat to group by, over what time window
outcomeOptionalComputed values: risk score, counts, aggregated fields
conditionYesThe final boolean over the declared event variables

Variables beginning with $ and bound to event fields are event variables ( $e ); those bound to a value used for grouping are placeholders ( $host, $user ). A placeholder appearing in match means "correlate events that share this value".

Module 3.2

Single-event vs multi-event rules

The distinction that unlocks correlation

Single-event: no match section; the condition is simply $e. One matching event, one detection. Good for unambiguous behaviour.

Multi-event: a match section declares the join key and window. This is how you express "the same host did A then B within ten minutes".

rule phish_to_takeover_chain {

  meta:
    description = "Web click to newly-seen domain followed by a login from a new ASN"
    severity    = "CRITICAL"

  events:
    // A: user browses to a newly registered domain (Netskope/proxy)
    $click.metadata.event_type = "NETWORK_HTTP"
    $click.security_result.category_details = /Newly Registered/ nocase
    $click.principal.user.userid = $user

    // B: successful login for the same user from an unexpected ASN
    $login.metadata.event_type = "USER_LOGIN"
    $login.security_result.action = "ALLOW"
    $login.principal.user.userid = $user
    not $login.principal.ip in cidr %corporate_egress_ranges

  match:
    $user over 30m

  outcome:
    $risk_score = max(95)
    $domains    = array_distinct($click.target.domain)

  condition:
    $click and $login
}

Counting and thresholds

  condition:
    #failed >= 10 and $success        // 10+ failures then a success
    #e > 50                            // event count threshold
    $a and not $b                      // A happened, B did not

#var is the count of events bound to that variable within the match window — the mechanism behind brute-force, spray and beaconing rules. Note also the not $b form: absence of an expected event (no EDR alert after a proxy block, no MFA event after a login) is often the most interesting condition you can write.

Sliding windows: match: $user over 30m evaluates continuously as events arrive rather than in fixed buckets, so a chain spanning a bucket boundary still fires. Keep windows as tight as the behaviour genuinely requires — wide windows increase both cost and coincidental matches.
Module 3.3

Outcome variables and risk scoring

Making alerts self-describing
  outcome:
    $risk_score = max(
        if($e.principal.user.userid in %executives, 90, 50)
    )
    $event_count   = count($e.metadata.id)
    $unique_hosts  = count_distinct($e.principal.hostname)
    $commands      = array_distinct($e.principal.process.command_line)
    $total_bytes   = sum($e.network.sent_bytes)

Functions available include max, min, sum, count, count_distinct, array_distinct, and conditional if() expressions. Two reasons to invest here:

  • Outcome values appear in the alert, so the analyst sees the command lines or byte totals without running a second search.
  • $risk_score drives prioritisation and can be raised contextually — the same behaviour on an executive or an admin account scoring higher than on a test machine.
Module 3.4

Rule lifecycle: test, retrohunt, tune, deploy

Using a year of history as your test set
1Prototype as a UDM Search until the logic returns what you expect.
2Convert to YARA-L and validate syntax in the rule editor.
3Run a retrohunt over weeks or months of history. This is the platform's best feature for detection engineering: you see the exact false-positive volume before the rule ever pages anyone.
4Tune using reference lists and narrow exclusions; re-run the retrohunt to confirm.
5Enable live with alerting on or off; decide run frequency and whether it creates a SOAR case.
6Version and review — rules are code; keep them in Git and deploy via API where possible.
Retrohunt is also incident response. When new intelligence lands — a compromised supplier, a fresh IoC set, a technique used against a peer — write the rule and run it backwards over the retention window. "Were we ever affected?" becomes a question you answer in twenty minutes rather than a quarterly project.
Module 3.5

Curated detections and IoC matching

What Google supplies, and how to treat it
  • Curated detection sets — rule packs maintained by Google covering cloud threats, Windows threats, and applied threat intelligence. Enable by set, tune by exclusion; you cannot edit them directly, so persistent noise is managed with rule exclusions and reference lists.
  • IoC matching — observed domains, IPs and hashes checked against Google and Mandiant intelligence, surfacing as separate matches rather than rule detections.
  • Risk Analytics — aggregated risk per entity from contributing detections, giving you "which user or host is worst right now" instead of a flat alert queue.

Treat curated content as a floor, not a ceiling. It cannot know your naming conventions, your admin tooling, your business processes or which of your users handle payments — and that context is precisely what makes a detection high-fidelity. Your own rules should encode the things only you know.

Drill 1

You want to detect "10 or more failed logins followed by a success for the same account within 15 minutes". Which YARA-L construct is essential?

match + count. Correlating separate events requires a join key and a window — match: $user over 15m — and the threshold is expressed as #failed >= 10 and $success in the condition. Outcome variables are useful additions but do not create the correlation; regex and reference lists filter, they do not correlate.
Tier 4 · Adversarialsenior

What the SIEM cannot tell you

Module 4.1

Failure modes and honest limits

The questions to ask about your own platform
  • Unparsed or partially-parsed sources — the field your rule depends on is empty for one product. Test per source, not per rule.
  • Silent feed failure — build a detection that alerts when a critical log type stops arriving. Nothing else will tell you.
  • Timestamp problems — sources with wrong clocks or timezone handling break correlation windows quietly.
  • Missing telemetry entirely — no rule compensates for command-line auditing being disabled, or an unmanaged device with no agent.
  • Parser drift — a vendor changes their log format and fields silently move. Periodically re-validate sample events for key sources.
  • Alert fatigue — a detection nobody triages is not a control. Measure closure reasons, not alert counts.
Holding an MSSP to account (SIEM edition): ask for the rule logic, not the rule name. Ask which UDM fields each rule depends on and confirm those fields are populated for the sources. Ask which of the log types have zero rules referencing them. Ask when each detection last fired and what the true-positive rate was. Ask what happens when a feed stops. These five questions separate a managed service from a subscription.
Drill 2

A rule for suspicious cloud API activity has never fired in six months. Most likely explanation to check first?

Assume the plumbing before the environment. A rule that has never fired is far more often broken than it is protective. Search the underlying UDM fields directly to confirm they contain data for the relevant log type, then generate the behaviour deliberately in a controlled way and confirm the rule fires. Record the date — that record is your validation evidence.
Referencesearchable

Glossary