SecOps ingests security telemetry, parses it into a single schema, indexes it against an entity graph, and runs detection rules continuously over both live and historical data. Retention is typically a year at a flat capacity price rather than per-GB-per-day, which changes behaviour: you stop rationing what you send and stop deleting history to save money.
| Method | Best for | Notes |
|---|---|---|
| Forwarder / Bindplane agent | Syslog, files, Windows events on-prem | A container or service you run; needs monitoring itself |
| Feeds | SaaS and cloud APIs (Workspace, AWS, Okta, Netskope) | Configured in the console; scheduled pulls |
| Ingestion API | Custom sources, in-house apps | Push UDM directly and skip parsing entirely |
| Native cloud integration | GCP, and direct exports | Lowest friction where available |
Every source is tagged with a log type (e.g. WINEVTLOG, AWS_CLOUDTRAIL, OKTA, NETSKOPE_ALERT, GMAIL_LOGS). The log type selects the parser, so getting it wrong is the usual reason fields arrive empty.
principal is invisible to every rule you write. Ingestion dashboards showing green volume are not evidence of usable telemetry.UDM describes every event with the same grammar. Learn these six top-level objects and you can read any event from any product.
| Object | Meaning | Typical contents |
|---|---|---|
metadata | About the event itself | event_timestamp, event_type, product_name, vendor_name, log_type, product_event_type |
principal | The actor | hostname, ip, user.userid, process.command_line, asset |
target | What was acted upon | hostname, ip, port, url, file.full_path, user, registry |
src | Source when distinct from principal | Often used in network and proxy events |
observer / intermediary | The device that saw or relayed it | Proxy, firewall, mail gateway |
security_result | Verdict | action (ALLOW/BLOCK), severity, rule_name, threat_name, summary |
network | Network specifics | dns.questions.name, http.method, ip_protocol, sent_bytes |
extensions | Domain-specific extras | auth.mechanism, vulns |
Read it as a sentence: principal did event_type to target, with security_result as the outcome. "User jdoe on host LAPTOP-01 (principal) performed NETWORK_DNS for cdn-metrics.io (target/network), result ALLOW."
| event_type | Fires for | Sourced from |
|---|---|---|
PROCESS_LAUNCH | A process started | EDR, Sysmon, 4688 |
NETWORK_CONNECTION | A connection was made | EDR, firewall, flow logs |
NETWORK_DNS | A DNS query | DNS servers, EDR, resolvers |
NETWORK_HTTP | A web request | Proxy/SWG (Netskope), WAF |
USER_LOGIN | Authentication attempt | IdP, Workspace, Windows, SSH |
USER_CHANGE_PERMISSIONS | Privilege modification | AD, cloud IAM, SaaS admin |
FILE_CREATION / FILE_MODIFICATION | File activity | EDR |
REGISTRY_CREATION / _MODIFICATION | Registry writes | EDR, Sysmon |
EMAIL_TRANSACTION | A message event | Gmail, email security |
RESOURCE_CREATION / _DELETION | Cloud resource lifecycle | CloudTrail, GCP audit |
SCAN_UNCATEGORIZED, GENERIC_EVENT | Everything else | Various |
USER_LOGIN with security_result.action = "BLOCK" covers Workspace, Okta, AWS console, SSH and Windows simultaneously. Writing the same detection five times per product is the cost you avoid by learning the schema first.Alongside events, SecOps maintains entities — assets, users, resources — assembled from context sources (asset inventory, IdP directory, EDR, vulnerability data). This is why clicking a hostname gives you an owner, an OS, recent alerts and prevalence rather than a bare string.
Entities are referenced in YARA-L via the entity keyword in a graph join, letting a rule ask "was the involved user, at the time of the event, in the finance group?" — context most SIEMs force you to hard-code into lists.
// operators
= != equality
> < >= <= numeric comparison
/regex/ regular expression match (add nocase)
and or not ( ) composition
// failed logins for one user across every product metadata.event_type = "USER_LOGIN" and principal.user.userid = "jdoe@example.com" and security_result.action = "BLOCK" // DNS lookups with long, high-entropy labels — tunnelling candidates metadata.event_type = "NETWORK_DNS" and network.dns.questions.name = /^[a-z0-9]{25,}\./ nocase // encoded PowerShell anywhere in the estate metadata.event_type = "PROCESS_LAUNCH" and principal.process.command_line = /-enc|frombase64string|downloadstring/ nocase // proxy uploads to personal cloud storage over a size threshold metadata.event_type = "NETWORK_HTTP" and network.http.method = "POST" and target.url = /dropbox|drive\.google|wetransfer/ nocase and network.sent_bytes > 10000000 // AWS: role assumption from outside known egress ranges metadata.log_type = "AWS_CLOUDTRAIL" and metadata.product_event_type = "AssumeRole" and not principal.ip in cidr ["203.0.113.0/24"] // aggregate: which hosts contacted a domain, and how often metadata.event_type = "NETWORK_DNS" and network.dns.questions.name = "cdn-metrics.io" group by principal.hostname
Two habits that speed everything up: search by event_type first to cut the corpus before adding conditions; and when a search returns nothing, check the field is actually populated for that log type before assuming the behaviour did not occur.
Reference lists are named collections — strings, regex patterns or CIDR ranges — usable in both searches and rules. They are the correct home for anything that changes independently of detection logic.
// in a rule or search
not $e.principal.ip in cidr %corporate_egress_ranges
$e.principal.process.file.full_path in %approved_admin_tools
$e.target.domain in %known_bad_domains
Good candidates: corporate egress and POP ranges, approved admin tooling paths, service-account names, break-glass accounts, executive accounts for high-touch rules, and vendor domains. Bad candidate: anything that encodes the logic of the detection rather than its data — that belongs in the rule so it is visible in review.
YARA-L borrows its shape from YARA but operates over time-ordered events rather than file contents. Every rule has the same skeleton:
rule suspicious_encoded_powershell { meta: author = "secops" description = "Encoded PowerShell launched by an Office application" severity = "HIGH" mitre_attack_tactic = "Execution" mitre_attack_technique = "T1059.001" reference = "internal-runbook-042" version = "1.1" events: $e.metadata.event_type = "PROCESS_LAUNCH" $e.principal.process.parent_process.file.full_path = /winword|excel|outlook/ nocase $e.principal.process.command_line = /-enc |-e |frombase64string/ nocase $host = $e.principal.hostname // placeholder for grouping outcome: $risk_score = max(85) $user = array_distinct($e.principal.user.userid) condition: $e }
| Section | Required | Purpose |
|---|---|---|
meta | Recommended | Documentation and ATT&CK mapping; surfaces in alerts |
events | Yes | Predicates the events must satisfy; declares variables and placeholders |
match | Multi-event only | What to group by, over what time window |
outcome | Optional | Computed values: risk score, counts, aggregated fields |
condition | Yes | The final boolean over the declared event variables |
Variables beginning with $ and bound to event fields are event variables ( $e ); those bound to a value used for grouping are placeholders ( $host, $user ). A placeholder appearing in match means "correlate events that share this value".
Single-event: no match section; the condition is simply $e. One matching event, one detection. Good for unambiguous behaviour.
Multi-event: a match section declares the join key and window. This is how you express "the same host did A then B within ten minutes".
rule phish_to_takeover_chain { meta: description = "Web click to newly-seen domain followed by a login from a new ASN" severity = "CRITICAL" events: // A: user browses to a newly registered domain (Netskope/proxy) $click.metadata.event_type = "NETWORK_HTTP" $click.security_result.category_details = /Newly Registered/ nocase $click.principal.user.userid = $user // B: successful login for the same user from an unexpected ASN $login.metadata.event_type = "USER_LOGIN" $login.security_result.action = "ALLOW" $login.principal.user.userid = $user not $login.principal.ip in cidr %corporate_egress_ranges match: $user over 30m outcome: $risk_score = max(95) $domains = array_distinct($click.target.domain) condition: $click and $login }
condition: #failed >= 10 and $success // 10+ failures then a success #e > 50 // event count threshold $a and not $b // A happened, B did not
#var is the count of events bound to that variable within the match window — the mechanism behind brute-force, spray and beaconing rules. Note also the not $b form: absence of an expected event (no EDR alert after a proxy block, no MFA event after a login) is often the most interesting condition you can write.
match: $user over 30m evaluates continuously as events arrive rather than in fixed buckets, so a chain spanning a bucket boundary still fires. Keep windows as tight as the behaviour genuinely requires — wide windows increase both cost and coincidental matches. outcome:
$risk_score = max(
if($e.principal.user.userid in %executives, 90, 50)
)
$event_count = count($e.metadata.id)
$unique_hosts = count_distinct($e.principal.hostname)
$commands = array_distinct($e.principal.process.command_line)
$total_bytes = sum($e.network.sent_bytes)
Functions available include max, min, sum, count, count_distinct, array_distinct, and conditional if() expressions. Two reasons to invest here:
$risk_score drives prioritisation and can be raised contextually — the same behaviour on an executive or an admin account scoring higher than on a test machine.Treat curated content as a floor, not a ceiling. It cannot know your naming conventions, your admin tooling, your business processes or which of your users handle payments — and that context is precisely what makes a detection high-fidelity. Your own rules should encode the things only you know.
You want to detect "10 or more failed logins followed by a success for the same account within 15 minutes". Which YARA-L construct is essential?
match: $user over 15m — and the threshold is expressed as #failed >= 10 and $success in the condition. Outcome variables are useful additions but do not create the correlation; regex and reference lists filter, they do not correlate.A rule for suspicious cloud API activity has never fired in six months. Most likely explanation to check first?