Two failure modes, both common. One organisation logs almost nothing and cannot reconstruct what happened. Another logs everything, pays enormously for it, and still cannot reconstruct what happened because the signal is buried and the retention on the useful sources was cut to control cost.
The interesting work is selection.
The question that drives it
What would you need to answer, six months after an incident, when the systems involved have been rebuilt?
Work backwards from the questions. Who authenticated, from where, when. What changed, by whom. What network connections existed between which hosts. What ran. What left the environment.
Sources that answer those questions are worth their retention cost. Sources that do not are worth much less than their volume suggests.
The sources that pay their way
Authentication. Every success and failure, from the directory, from the VPN, from the cloud, from privileged access. This is the backbone of every investigation.
Privileged command execution. What administrators actually did.
Configuration change. From your infrastructure, your network devices, your cloud control plane. Ideally from version control, which gives you this for free with attribution.
Network flow records. Not packet capture. Flow data is compact and answers the lateral movement question, which is the question you will have.
DNS queries. Small, and disproportionately useful for identifying compromised hosts.
Endpoint detection telemetry, if you have it.
The sources that are mostly volume
Application debug logging at scale. Verbose web access logs beyond a modest retention. Health check traffic. Anything a system emits continuously that describes normal operation with no security relevance.
These are useful operationally, briefly. They do not need to be in an expensive analysis platform for a year.
The two-tier pattern
Send everything to cheap object storage, and only the security-relevant subset to the expensive searchable platform.
Cheap storage keeps the full record, at a cost per terabyte that makes long retention affordable. If an investigation needs something that was not indexed, you can retrieve and load it — slower, and available, which is the difference between an incomplete investigation and a slow one.
This one architectural decision resolves most of the tension between coverage and cost, and it is the thing I would implement first.
Retention by value
Authentication and change records: long, because the questions arrive late. A year at minimum, longer where regulation says so.
Flow and DNS: medium. Months rather than years.
Operational noise: days or weeks.
Set these deliberately rather than applying one policy to everything, which is how organisations end up paying for a year of load balancer health checks.
Two things that are not optional
Time synchronisation. Correlating events across systems requires clocks to agree. This sounds trivial and it is the reason a surprising number of investigations stall.
Logs outside the compromised environment. An attacker with administrative rights deletes logs. If your logging platform authenticates against the same directory as everything else, it is inside the blast radius. Ship logs somewhere with separate credentials, ideally append-only.
That second point is the one most often missed, and it converts your entire logging investment from evidence into something the attacker controls.
The check
Pick a system. Ask: if this were compromised today and rebuilt tomorrow, what could we reconstruct in six months?
Do it for three systems. The gaps you find are your logging roadmap, and it will be shorter and cheaper than the one a vendor would sell you.