Data collection and evidence gathering made easy
As businesses face continued disruption, cyber threats persist, as does the ongoing need to protect data. For IT Consultants, travel to sites to conduct security audits and maturity assessments has been constrained, making traditional service delivery methods impractical.
The internal security audit tool, the Essential 8 Auditor systematically undertakes the data collection and evidence gathering elements of the security audit process with a single click. It also has the functionality to save audit definitions, enabling users to run like-for-like audits quickly.
How the Essential 8 Auditor supports security audits
Audit and assessment processes involve several steps and the measurement of a number of ever-changing variables. Depending on the focus of an audit and the goal, the shape of the tasks might be different.
The way an assessment is performed, the depth and breadth of evidence gathering and the analysis and reporting, needs to accurately reflect an organisation’s risk exposure. A typical audit process will involve these five stages:
Security Audit Process
Five areas where the security audit process is transformed
Stage 1 – Planning, Scoping and Logistics
The Essential 8 Auditor reduces site visits and travel. It also increases the breadth of audit scope.
The planning part of a security audit will define the objectives, scope, work activities and logistics. This is where a balance has to be drawn between coverage and budget, and how best to efficiently deliver the outcomes. Decisions will need to be made on:
- The scope of the audit: What parts of the environment, what controls, the balance between technology, policies and processes and what activities will be involved.
- The audit method: The combination of manual assessments, onsite visits, interviews, questionnaires, audit tools and data/evidence requirements.
- Scheduling of the audit: Including the assignment of resources and the planning of site visits, inspections, meetings and the associated travel.
- Planning for the impact of the audit: On any operational systems and the involvement and collaboration with operational personnel.
If assurance in the outputs can be gained with reduced site visits then the cost in time and travel goes down, and if more accurate ways of collecting data can be used then the subjectivity and vagaries of interviews and questionnaires can be avoided. If the most relevant and highest impact controls can be prioritised, then the scope of coverage could be broader for an equivalent amount of budget/effort.
The other risk in the planning of any security audit is the potential for selection bias – deciding to audit certain systems or sites because they are easier or because they are known to have data available. For example, choosing to conduct an audit externally with scanning tools because going to site is more expensive, or choosing controls that are well managed and generate detailed metrics instead of ones where metrics and records are harder to obtain.
The deciding factors here are the tools and facilities available as they will drive what site attendance is required, what data can be obtained, and how wide a scope can be covered for a given set of controls.
Stage 2 – Data Collection and Evidence Gathering
The Essential 8 Auditor systematically gathers audit data and evidence of successes and failures.
The collection of data, metrics and audit evidence, the “fieldwork”, is the most resource intensive phase of the audit process. It’s the easiest one to make mistakes in, omit key facts or rely on assumptions that are not really valid. The data should be gathered across as wide a scope as possible, ideally the entire environment. Manual approaches often rely on sampling and this can introduce biases or omissions.
Types of fieldwork and data collection
One obvious goal is to optimise the amount of labour involved in doing this; by minimising the time spent querying systems, gathering data and creating reports reports or spreadsheet extracts. This means that more systems and controls can be covered. Doing this programmatically also means the data collection can be done by less experienced or non-expert staff, or by local IT administrators. Taking data direct from the source also removes interpretation and subjectivity from the collection process around “how compliant” or “how well implemented” a control or system is.
Expert analysis of the collected data then benefits from better, more accurate and comprehensive inputs and less reliance on or overhead from travelling, collecting, collating and extracting – hence more time to understand and draw conclusions or examine areas of the cyber defence posture that are not embedded in technology.
The Essential 8 Auditor:
1. Confirms the presence of controls or the implementation of policies
2. Gathers evidence and records of their operation or execution
3. Identifies weaknesses, failures or omissions
4. Pinpoints failures and enables the analysis process to link these to identifiable root causes
Stage 3 – Analysis and Interpretation
The Essential 8 Auditor accurately and consistently calculates results, analyses objectively and identifies patterns.
The analysis of collected audit and control data can be split into two elements:
Mathematical or Statistical analysis
Using a large data set covering a large number of systems, controls and degrees of implementation requires some of the analysis that is best undertaken by computers. It is important that the toolset makes the calculations (i.e. produces outputs), rather than allowing the consultant or auditor to make the calculations (much as a spreadsheet would).
If a set of results will benefit from having a count of those values that meet certain criteria, or the minimum/maximum being derived, or the average – or the percentage equivalents of these – then the data collection/analysis process can derive this and feed it into the outputs. This will avoid someone having to manually operate a tool to filter the data into subsets, calculate totals and pivot them to derive the data representation they need.
Interpretation and expert analysis
The meaning of control failures, the omissions of implementation, the numbers of affected systems or degrees of compliance are all things that consultants, auditors or security experts, often with a view of the wider context around IT operations or business objectives, can best understand. This is where the consultant or auditor interprets the data (raw and processed) to derive the findings, insights and recommendations that the customer or end-user needs.
The performance metrics that result from the analysis of data from controls can be used to provide a view of cyber posture or residual risk and security operational performance that an auditor, manager or consultant can put into context for the business to use in decision-making.
The Essential 8 Auditor undertakes a wide range of calculations to summarise, analyse and identify patterns in the data.
The derivation of performance metrics from data through calculation can be automated so that the expert can focus on understanding what these actually mean, rather than trying to derive them in the first place.
4 – Reporting
The Essential 8 Auditor enables you to distribute performance metrics to all stakeholders easily.
Every consultant, auditor and expert will have a way to report and present the findings and outputs of their work and the interpretation they have made as to the state of the cyber risk strategies that the controls have enforced.
This analysis and reporting is the culmination of the report author’s wisdom and experience, and it must be attuned to the audience for which it is intended.
Managers and risk owners need to understand the coverage, effectiveness and maturity of the controls that are in place, and the risks that their business or data is exposed to as a result of this. Hence the graphical representation of KPIs and the ability to understand how this translates into a maturity score that combines quality, coverage and effectiveness is vital. This audience doesn’t want lists of issues or pages of detailed analysis, they want business focused performance metrics and dashboard type outputs that show them the current posture and the trends over time, to help them understand if it is getting better or worse.
Those tasked with remediation activities or the prioritisation of improvement projects have a different output requirement, they need to know the specifics of which systems, controls or failures were detected so they can group these together to resolve them or put in place work orders or change controls to address the issues. This form of output will often be the input to other data sets or systems; hence the appearance matters less, in fact the rawer the data is the better.
The Essential 8 Auditor presents easy to understand KPIs and outputs in dashboard form and a view of maturity that gives business unit owners, risk and compliance teams and the C-suite the information they need, at a glance, to understand the risks and the performance of the security function.
The technology can generate raw outputs for technical use, or for other systems that need lists of systems or control failures to be remediated.
5 – Remediation
The Essential 8 Auditor automatically verifies that key issues have been addressed.
Remediation is not a part of the security audit process itself, but it is a key element of the business improvement cycle that the audit initiates. Once findings have been made the relevant operations or technical functions must prioritise and resolve the issues.
This may be fixing point problems, investing in technology or addressing route causes so that expected controls and safeguards operate as intended.
The important part of this process is the re-evaluation and reassessment of the controls’ effectiveness and performance metrics AFTER the work has been conducted.
Here the use of technology and an optimised audit process is particularly useful. If the checking of the previous audit findings necessitates a complete rerun of the process – the site visits, field work, recalculation and reporting – it is unlikely to be viable. You are effectively doing the audit twice.
Where technology has been put in place to collect and gather data, to analyse it and present the identified issues and performance metrics and control maturity, then it becomes a much simpler exercise to rerun.
A full audit with root cause analysis is unlikely to be needed in this instance, as the new and improved scores and the greatly reduced lists of identified problems, are sufficient output to show improvement.
The Essential 8 Auditor verifies that remediation steps have been completed. The first pass will almost certainly identify findings, so a retest or follow-up check should be part of the planning.
Get results fast
The Essential 8 Auditor can be deployed on site or a remote network, without the need for a consultant or audit and risk manager to visit that site. No security expertise is needed to deploy the solution; it can be installed and run by a network or system administrator locally. The gathered data can be reviewed off-line and extracted for additional analysis or report preparation.
Want to find out more?
If you’d like to request a demo, find a reseller or ask a question, please contact us by clicking on the button below.