When you do a security assessment, you need to elaborate some recommendations to mitigate the potential risks. This is one of the most difficult parts, because bad assumptions can easily lead to false sense of security and overspending…
Security Risks are everywhere, by example, when you do a penetration testing, you discovery a lot of potential security risks, and you combine the business experience with the external auditor experience to create an objective view of the problem.
So the ulterior purpose of this thing is to mitigate the risk to an acceptable point.
What does mean mitigation?
Mitigation means to make it difficult enough for the adversary to exploit us, but in general is a complicated equation of things that involves:
- The mitigation costs should not exceed the expected ROI from the business:
- It can be tricky if the protected organization is not a profitable organization (E.G. gov page)
- If the mitigation would cost more than other possible business, then, maybe it’s also a bad mitigation, the business owner can decide to switch to another thing
- Sometimes the reputation risk of being attacked/breached is enough by itself to justify the mitigation cost
- Usually the customer/user data relevance/privacy should be considered
- And also several laws should be considered to choice the proper mitigation.
- The mitigation should be good enough to make the attack costly enough to the adversary to find another victim, but not necessarily:
- Sometimes the adversary encounter an opportunity in time for a vulnerability (E.G. during an APT)
- Sometimes the attack is not driven by money or maybe the adversary have some fixation with you (E.G. public institutions, another countries)
- And also, sometimes, adversaries can invest money on the attack (0days exploits, insiders…)
- Not because you think that some potential attack is difficult, it will be difficult for your adversary.
- The proper mitigation should not only mitigate the discovered risk, but also future potential risks associated with your way of doing things…
Not because you think is difficult…
Not because you think something is difficult, it will be difficult for the adversary. At this point, I can bring some examples:
Most of the pentesters are good enough to exploit common vulnerabilities in very standard environments (exactly what it is in the ethical hacking crash course), but if the environment changes a little bit (e.g. an SQL injection is found inside an encoded base64 field, or inside the uploaded file metadata), the scanner and the “expert” will fail to address some vulnerabilities and potential risks.
This happen so often even with 10-years experts… there is no easy way to avoid it.
Remember, there is a world outside, the world is not homogeneous, in fact is very heterogeneous, and there are people better than us there that can develop exploits very quickly… And there is also a black market for all type of things/artifacts that are not available to your local pentester…
Given that, I will give you an EXTREMELY important advice:
Never say that someone will have to work 10 months on some vulnerability because you feel is complicated , specially if you don’t have a non-biased mathematical proof..
By example, in cryptography we know “for sure” that some algorithm requires n-trillion years to be broken (at the current available processor power) and this is because there is a mathematical proof for that, but to say that you need 3 months to elaborate an AV bypass or an ASLR bypass to exploit a known vulnerable server? just because you are not able to do it? just because you are not expert on it? that will be a very unfortunate bad assumption…
Moreover… I saw auditors doing mistakes like this: they try to put a price to the hacker time, and that will cost the same (or more) that the auditor time, and the whole idea/bet is that hacker will not spend his valuable time in you, because it’s too expensive?
bad idea my friend…
Hackers could be motivated by money, or may be not, sometimes they are just there because it’s funny (something that many can’t understand because you are too professional to understand this).
But let’s suppose that money is the root of all evil, you know that there are good hackers in countries where 400usd per month is their average salary (not per hour)?
So, the point is: don’t play with the risk scores because when don’t have the knowledge…
And not in the time of the assessment…
And no, because your 200usd/hour pentester have not breached you in 5min, does not mean that it will be impossible for anyone else to do that in a non-restricted time span…
The relation is quite simple:
More hours => More Discovered Vulns/Potential Risks
And because you have not considered enough hours for testing/scan/analysis, does not mean that you don’t have the risk…
To complicate the things a little bit more… everyday there is a new vulnerability out there (that may have not been addressed in the last report)… and every so often you may be releasing new things/features/data that have been not properly audited…
Same test, same methodology, different results:
You should know (and it is very annoying to know) that every test may start in the same way/methodology and result in different discovered vulnerabilities, That is not because the test was not have be done properly, but because skills (besides of the methodology) plays a very important role here… each pentester have a different skill set, so even with the same methodology the result WILL be different…
To try to look that there is no entropy and make this look like a science rather than an art, most of the vendors try to assure that the methodology is everything… but, in the reality is not like that… anyone saying that is not a trusted provider.
Even a little different change in the skill set can deliver different results…
I can give many examples, but the point is, that in general we have a limited set of hours to do the job, so the pentester have to optimize the audit based on his own experience (choose what to do), and also the audited technologies are in constant change, each technology is very rich on their own, and every technology combination is unique and produces unique results…
methodology usually gives you guidelines on how to work, gives you the most common vulnerabilities tests that you may execute (e.g. sql injection, xss…), but the methodology can’t define what you exactly have to do on a reduced time span… The experience would…
- Determine what is a potential security issue or what is not (subjective to the pentester)…
- Determine what tests are going to be discarded or included (subjective to the pentester)…
- E.G. you usually don’t try to exploit log4j in a php environment, there is virtually no any methodology that tells you that, but maybe you decided to spend some time on it just because you heard that some of the sysadmins is a very known java activist, the site is very slow, and they may be combining things… bingo, you found a vulnerable home made analytics thing made in java over a php site… What is the chance that another pentester will found this only based in the methodology and not implying that you will have to do a 5 years activity to test virtually every non-related exploit code on each field…
- If there is not enough time to crack every possible password, you may be using a statistical method like markov chains to crack it, and feed it with what the pentester consider is the best for that… (subjective to the pentester)…
- Try to understand the defense itself and how to bypass that (eg. creating a new method/technique, relative to the pentester)
- And more…
My recommendation: hire different skilled pentesters in each iteration, rotate them, and NEVER assume that if the last one have not discovered something, then that thing is mitigated…
To discover if something is mitigated, it’s a very complex/difficult thing, you should have some proofs like code proofs and behavior proofs…
NEVER measure the risk based on your current mitigation policy
Another tremendous error when doing assessments is to start in blind behind the high level of protection that you can… If you think that testing should be oriented against the defenses to proof that your million dollar security is good enough, you are just wasting your money… I would prefer to grant me exceptions under the contract to avoid these systems from the beginning, because if not…
The pentester can be consuming dozen of hours fighting against WAF, Antivirus, and so on… and the common results (with good pentesters) are:
- Antivirus software bypassed
- EDR/XDR bypassed
- Logs compromised
- WAF policies not aligned with the real app risks (E.G. lot of unresolved IDOR’s)
- Password centralization platform compromised…
beside of that, spent hours defusing the perimeter are not going to be used to determine the real vulnerabilities (patches, outdated systems, policies)…
My recommendation is:
the most efficient and effective approach is to find possible vulnerabilities/risks is from the inside, without the armor, and from there, you put the armor outside to protect on what you can’t see…
Raise the barrier enough to discard the n00bs, but not enough for the APT…
Fighting against SQL injections or XSS is relatively easy… mitigating the thing for the n00b can be easily done… but don’t stop there…
When you have mitigated all of this things… more advanced stuff/vulns are waiting there too to be mitigated, and guess what: there are people outside with the advanced exploit exclusively working on “mitigated” environments.
Why?
Because they prefer a site that has not been compromised before… no competition, no tainted environment, less alert state in the company…
False sense of security (the risks)…
Bad assumptions will drive you to the worst state: “False sense of security”
… but why worst?
- Overspending or Spending money on things you don’t need
- Not spending in things that you actually need
- and the most terrible phrase: I’m secure…
The most awful things happens when you say:
I’m secure
That horrible phrase is going to cause you to avoid any contingency plan and any willingness to prove the false… That will translate on every type of hidden risks…
The defense should be done in layers… specially when you can’t see your own problems… this is the key
The proper mitigation will not only mitigate the discovered risk, but also future potential risks associated.
The conclusion is very simple… you need to make a plan to mitigate, the plan should be based on the root of the problem, not on each vulnerability per se… and that plan have to accept that maybe the found vulnerabilities are not all of them, and maybe some vulnerabilities have been underestimated…
When you accept that, you understand that the report itself is not valuable because of each detected vulnerability, but because it can help you to identify the main issues: eg. updates, policies, code safety, etc…
Aaron (unmanarc) is a security researcher with more than 17 years of experience in penetration testing…