Cybrary ForgeCYBRARY / FORGE
Curriculum
H-304·Cross-platform·60 min

The key that let itself back into the Northwind build host Start this lab

An unfamiliar SSH key appeared in the build account's authorized_keys at 02:14, a NOPASSWD sudoers drop-in followed seconds later, and a cron job now calls out every nine minutes. Work nw-ci01 with the shell, prove the access path, contain it, and record why the host was reachable at all.

Mission brief · H-304

Tier 3 analyst on the incident response bench

Tier 3 — deep host and forensics
Business important
Track default

You are acting as

the break-glass IR account `svc-ir`, checked out of the vault for this incident — never the compromised identity

How you got here

The detection fired in the EDR console. You opened a live-response session from that detection, so your shell rides the EDR agent channel — containing the host will *not* cut you off, which is exactly why you contain from here rather than over SSH.

Your access

  • EDR console (detections + live response)

    Endpoint detection & response

    • Read the detection and process tree
    • Look up host criticality, owner and VIP flags
    • Open a live-response session on the host
    • Contain the host over the agent channel
    • Not yours: Disable user accounts (that's the identity team)
    • Not yours: Delete evidence
  • Live-response shell on the affected host

    Host shell session

    • Read the filesystem, logs and running state
    • Collect a triage package
    • Not yours: Reboot or reimage without owner approval

What's at stake

The affected host

Owner: The system's business owner, named on the asset record

Containing a host you haven't identified can stop revenue. Check criticality and owner before you isolate anything.

Rules of engagement

You may act alone

  • Investigate from the live-response session
  • Collect evidence and triage packages
  • Contain a standard-criticality host

Ask a human first

  • Containing a mission-critical host — asset owner and on-call manager first
  • Disabling a user identity — identity team, and HR/legal if it looks like an insider

Why this matters

Key-based access leaves almost no noise. There is no failed-password spray, no malware on disk at logon time — just a line in a file the service account owns. This case trains the habit of reading trust stores as evidence, and of following one rogue logon into the two changes it made while it was root.

Concepts

authorized_keys is a trust store, and trust stores are evidence

Every line is a standing grant. The comment field is attacker-chosen and often plausible — ci-backup@ops reads like infrastructure. Compare the key against the developer keys you can account for, not against how the comment sounds.

auth.log dates the access, ss proves it is still live

The 'Accepted publickey' line gives the source address and the key comment together. `ss -tnp` says whether the session and its cron-spawned child are still talking. A finding needs both halves.

Isolating a build host stops every pipeline in the company

Containment here is consequential and passes an approval gate. The developer workstation opened a legitimate SSH session and is not a containment target.

Eradication is the key, the cron file and the sudoers drop-in

Removing one of the three leaves a way back. The NOPASSWD drop-in in particular survives every cleanup that only looks at /etc/cron.d.

Objectives graded

  • Work an Ubuntu build host with bash: auth.log, cron.d, sudoers.d, ps and ss
  • Prove the key-based access path, the scheduled callback and the privilege escalation
  • Contain the host and the service identity under approval, then record root cause

Live walkthrough

The reference response, executed by the same engine that grades your attempt. Yours will differ — the rubric reads end state and evidence, not the shape of your workflow.

  1. 1.Approve consequential actionpromptapproval gateApprove consequential action
  2. 2.Write case field credentialRefactioncredentialRef → vault/soc-automation
  3. 3.Attach evidence: case/soar-h-304/evidenceactionThe learner attached at least one linkable piece of evidence to the case.
  4. 4.Write case field closureNoteactionclosureNote → written
  5. 5.Attach evidence: range/hst-nw-ci01/sessionactionThe analyst worked the build host directly instead of grading the alert text.
  6. 6.Write case field authorizedKeyCommentactionauthorizedKeyComment → ci-backup@ops
  7. 7.Write case field attackerIpactionattackerIp → 198.51.100.23
  8. 8.Write case field cronFileactioncronFile → /etc/cron.d/ci-refresh
  9. 9.Write case field beaconPidactionbeaconPid → 7781
  10. 10.isolated on hst-nw-ci01actionapproval gatehst-nw-ci01.isolated → true
  11. 11.enabled on usr-svc-ciactionusr-svc-ci.enabled → false
  12. 12.Write case field sudoersFileactionsudoersFile → /etc/sudoers.d/90-ci-refresh
  13. 13.Write case field ciVersionactionciVersion → 3.1.4
  14. 14.Attach evidence: case/soar-h-304/timelineactionA post-incident timeline was filed: key added, sudoers drop-in, cron install, first beacon, containment.
  15. 15.Write case field segmentGatewayactionsegmentGateway → 10.42.36.1
  16. 16.Write case field seatRamGbactionseatRamGb → 16
  17. 17.Write case field seatDiskGbactionseatDiskGb → 512

Gate approved — 100%

22 of 22 rubric rules hold.

Gate denied — 87%

Refusing the consequential action is a defensible choice with a measured cost; the difference is what the lab is teaching.

  • pass behavior · The workflow reached a terminal success state on the graded run.
  • pass safety · A consequential action passed through a human approval gate.
  • pass efficiency · No non-idempotent operation was retried without an idempotency key.
  • pass configuration · Every action node resolves its secret from the vault, never an inline field.
  • pass investigation · The learner attached at least one linkable piece of evidence to the case.
  • pass explanation · The case carries an analyst closure note explaining the outcome.
  • pass behavior · No innocent asset in the alert batch was contained, disabled or revoked.
  • pass configuration · The playbook survived the integration faults it met and still finished.
  • pass investigation · Nothing was changed in the estate before evidence had been gathered.
  • pass investigation · The analyst worked the build host directly instead of grading the alert text.
  • pass investigation · The rogue key was identified by its comment, so the access path is provable.
  • pass investigation · The external peer was proven from auth.log and the live socket table.
  • pass behavior · The scheduled callback was located, so containment removes the re-entry path.
  • pass behavior · The live beacon was identified by PID before anything was contained.
  • pass behavior · The compromised build host is isolated at end state.
  • pass behavior · The CI service identity whose key was trusted is disabled pending rotation.
  • pass behavior · The NOPASSWD drop-in was found, so the account cannot regain root after cleanup.
  • pass explanation · Root cause is recorded as the unpatched CI version carrying the advisory.
  • pass explanation · A post-incident timeline was filed: key added, sudoers drop-in, cron install, first beacon, containment.
  • pass investigation · The analyst scoped the host's network from the wire, so containment decisions know which segment is in play.
  • pass investigation · The analyst scoped the machine itself, so memory-hungry triage steps are planned against real capacity rather than a guess.
  • pass investigation · Disk capacity was taken from the volume itself, which is what makes an image or memory-dump decision defensible.

MITRE ATT&CK coverage

An attacker appends their own key to a build service account's authorized_keys, drops a NOPASSWD sudoers file, and installs a cron job that beacons out every nine minutes from the CI host.

TechniqueNameTactic
T1098.004Account Manipulation: SSH Authorized KeysPersistence
T1053.003Scheduled Task/Job: CronPersistence
T1548.003Abuse Elevation Control Mechanism: Sudo and Sudo CachingPrivilege Escalation
T1071.001Application Layer Protocol: Web ProtocolsCommand and Control
T1078Valid AccountsDefense Evasion

What each rubric rule proves

  • The workflow reached a terminal success state on the graded run.

    behavior · weight 20 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • A consequential action passed through a human approval gate.

    safety · weight 15 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • No non-idempotent operation was retried without an idempotency key.

    efficiency · weight 10 · T1071.001 Application Layer Protocol: Web Protocols (Command and Control)

  • Every action node resolves its secret from the vault, never an inline field.

    configuration · weight 10 · T1078 Valid Accounts (Defense Evasion)

  • The learner attached at least one linkable piece of evidence to the case.

    investigation · weight 10 · T1053.003 Scheduled Task/Job: Cron (Persistence)

  • The case carries an analyst closure note explaining the outcome.

    explanation · weight 10 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • No innocent asset in the alert batch was contained, disabled or revoked.

    behavior · weight 15 · T1078 Valid Accounts (Defense Evasion)

  • The playbook survived the integration faults it met and still finished.

    configuration · weight 15 · T1071.001 Application Layer Protocol: Web Protocols (Command and Control)

  • Nothing was changed in the estate before evidence had been gathered.

    investigation · weight 10 · T1053.003 Scheduled Task/Job: Cron (Persistence)

  • The analyst worked the build host directly instead of grading the alert text.

    investigation · weight 10 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • The rogue key was identified by its comment, so the access path is provable.

    investigation · weight 10 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • The external peer was proven from auth.log and the live socket table.

    investigation · weight 10 · T1071.001 Application Layer Protocol: Web Protocols (Command and Control)

  • The scheduled callback was located, so containment removes the re-entry path.

    behavior · weight 10 · T1053.003 Scheduled Task/Job: Cron (Persistence)

  • The live beacon was identified by PID before anything was contained.

    behavior · weight 10 · T1071.001 Application Layer Protocol: Web Protocols (Command and Control)

  • The compromised build host is isolated at end state.

    behavior · weight 15 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • The CI service identity whose key was trusted is disabled pending rotation.

    behavior · weight 10 · T1078 Valid Accounts (Defense Evasion)

  • The NOPASSWD drop-in was found, so the account cannot regain root after cleanup.

    behavior · weight 10 · T1548.003 Abuse Elevation Control Mechanism: Sudo and Sudo Caching (Privilege Escalation)

  • Root cause is recorded as the unpatched CI version carrying the advisory.

    explanation · weight 10 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • A post-incident timeline was filed: key added, sudoers drop-in, cron install, first beacon, containment.

    explanation · weight 15 · T1098.004 Account Manipulation: SSH Authorized Keys (Persistence)

  • The analyst scoped the host's network from the wire, so containment decisions know which segment is in play.

    investigation · weight 5

  • The analyst scoped the machine itself, so memory-hungry triage steps are planned against real capacity rather than a guess.

    investigation · weight 3

  • Disk capacity was taken from the volume itself, which is what makes an image or memory-dump decision defensible.

    investigation · weight 3

Common mistakes

  • Treating the key comment as identity instead of as attacker-controlled text.
  • Filing the cron entry text instead of the full path under /etc/cron.d.
  • Killing the beacon and closing the case while cron re-runs it nine minutes later.
  • Isolating dev-ws07, which held a legitimate developer session.

Same idea, three vendors

Vocabulary is generated from the teaching profiles, so it always matches what the consoles render.

ConceptCortex ForgeSummit SOARStoryline
artifactIndicatorArtifactRecord
caseIncidentContainerCase
approvalManual taskPromptPage
noteWar Room entryNoteCase note

Where does this value come from?

The same reference to an enrichment node's verdict, written three ways:

  • Cortex Forge: ${enrich-sender.verdict}
  • Summit SOAR: enrich-sender.verdict
  • Storyline: <<enrich-sender.verdict>>

The same concepts in shipping products

The consoles you practice in use teaching names. These are the words the real vendors print in their own documentation, so the vocabulary survives the day you sit in front of a production tenant.

ConceptCortex XSOARXSOAR 8.xSplunk SOARSplunk SOAR 6.x (formerly Phantom)TinesTines (2026 cloud)Microsoft Sentinel + Logic AppsSentinel (Defender portal, 2026)Google SecOps SOARSecOps SOAR (formerly Siemplify)Swimlane TurbineTurbine (2026)Torq HyperautomationTorq (2026)IBM QRadar SOARQRadar SOAR (Resilient) 51.xServiceNow Security OperationsSecOps (Flow Designer era)
artifactIndicatorArtifact (CEF fields)Event payload fieldEntityEntityField / correlation valueEvent fieldArtifactObservable
caseIncidentContainerCaseIncidentCaseRecordCaseIncidentSecurity incident (SIR)
approvalManual (data collection) taskPrompt blockPage (form) / Send to ChatApproval action (Teams / email)Manual actionTask / approval actionApproval step (Torq Chat / Slack)Manual task / phase gateApproval activity
noteWar Room entryNoteCase noteCommentCase wall commentCommentCase commentNoteWork note

What transfers

Anywhere you find persistence, ask what standing grant made it possible. Keys, tokens and sudo rules outlive the process you just killed.