Technical Training Done Wrong
Last Updated on Monday, 21 June 2010 23:07
Written by Brian Baskin
Friday, 18 June 2010 01:32
There's very rarely a large-scale news event that takes my two primary jobs: information security and teaching, and touts them in front of the public. So, here's a bit of the inner workings of my mind for my day job, and how amateur marketing-driven training casts a large shadow for the entire industry.
In recent weeks that has been an ongoing discussion over the marketing tactics used by a small information security firm, LIGATT Security. More truthful details on their company can be found elsewhere, such as attrition.org and Catch 22. My focus is instead on the initial project that caused a huge surge of discussion over LIGATT: Their Learn to Hack in 15 Minutes campaign. And while this will focus on the efforts by LIGATT, it is also a note about many forms of training being given by novices.
When I saw notice of the campaign I physically cringed. I knew it was going to fail and be mocked. And, it did.
This form of training takes the form of abbreviated tweets on Twitter from the @LIGATT profile. Examples include:
"Lesson 2: Footprinting: The next step of the information gathering process is to try to identify the range of IP addresses the target uses. via web"
"Lesson 2: Footprinting: Nslookup is a program to query domain name servers. This information can be used to diagnose the DNS infrastructure. via web"
"Lesson 3: Scanning: The goal of the scanning phase of pretest reconnaissance is to discover open ports and find vulnerable applications. via web"
"Lesson 4:Hacking Techniques:A pentest is more about taking the view of a hacker by seeing what can be accomplished and with what difficulty. via web"
"Lesson 4:Hacking Techniques:Cross Site scripting is created by failure of Web-based app to validate user input before returning to client. via web"
There are a number of issues with this style of training, too many to list here. The ultimate result is that you receive training that is not training at all: it's just a series of one-off nuggets with no cohesion and no actionable intelligence.
And it's just the latest example in the cyber training rage that everyone seems to be undertaking. Over the last 10 years, I've seen many large corporations start up their own training divisions that focus around the products and services that they provide. And they all take the same, simple approach. "Let's write up all this information and sell it to others. And every time we update the project, they have to be trained all over again! We'll be rich!" The result, though, is expensive, substandard training that does not educate.
Proper education takes effort, and it takes time. To understand this, let's talk about formal education in the cyber security industry... my favorite topic :) For over 10 years I've worked professionally as a cyber security researcher and technical training professional teaching cyber crime and information security content. I've developed college-accredited courses on incident response, Linux intrusion analysis, undercover Internet investigations, and large-scale intrusion investigations. I'm even a Certified Technical Trainer (CTT+) :)
What's wrong with this approach?
To sum it up, everything. This type of training takes a large volume of content, whittles it down to a small (140 char) paragraph, and sends it to a largely unknown audience. There has been no assessment to the type of audience it is intended for. There's no regular reviews of the material. There's nothing actionable in the tweets themselves. It's like learning from hacker flashcards: good if you need to memorize data quickly for a short period of time, but useless for long term cranial storage.
Doing it right
So, let's go through a scenario where I would develop similar training. How can security training be done properly to provide the best education for the students while remaining quick, efficient, and inexpensive. Truth be told, I've done this. Along with Johny Long and Marcus J Carey, we developed a two-day "Hacking Stuff" course that provided hands-on, realistic intrusion training to beginners.
I develop training based upon the ISD (Instructional Systems Design) ADDIE approach. ADDIE breaks down the process into five stages: Analysis, Design, Development, Implementation, and Evaluation.
Interviews, surveys, meetings, discussions: these all have to occur before anything is written. Who is the target audience? What is their background? Are they security laymen just entering the industry? Are they corporate leadership with no technical skills but large budgets? Are they consumers that purchase products that need to be secured?
What is the delivery platform? What are the strengths (immediate, free, vast audience) and weaknesses (140 char tweets) of it.
If you screw up this step, the course will fail.
Evidence that this stage was lacking in the LIGATT training was evidenced by one of their early tweets:
"How to be Hacker. For some of you who are experts at hacking, the beginning my be slow for some of you & to fast for others. via web"
A "one size fits all" approach to training does not work, at all, for anyone. If you shoot too high or too low, you'll lose your audience. They'll either be lost by the large amounts of technical information, or they'll be offended at how basic the material is. You need to pick your desired audience, market solely to them, then aim just slightly over their heads. Close enough that they understand the basic concepts and terminology, but just a bit out of reach so they have to exercise their synapses to connect the dots.
In the design stage you take the information learned from Analysis and start setting up your boundaries. What are you high-level and low-level objectives that students need to learn? High-level objectives would be based upon the stages of an attack, such as: Reconnaissance, Intrusion, Advancement, Entrenchment, Exfiltration.
These would then be broken down into low-level objectives. Reconnaissance would include Open Source Intelligence gathering (OSINT), vitality scans, and port scans, for example. Some of these can then be broken down into further objectives. OSINT would include Google Hacking, Maltego, social network scanning, etc.
All objectives are laid down into a logical order and structure (think "Table of Contents") and reviewed to ensure it is a natural progression of knowledge. Objectives should build upon each other.
The development stage is the heavy one - all of the material is developed here. The material is all originally developed (key phrase there). Any material that is to be cited is flagged with its original source so that releases can be obtained later. Enough said.
Scenarios have to be drafted up. Fictitious business names, locations, email addresses, and accounts have to be created. Virtual servers are setup to simulate a real working environment. Custom outfitted systems for students to hack from and virtual servers to act as targets. WebGoat is a cool idea, but is an amateur cop-out for professional training. The environment should mirror your students' own environments as close as possible. A triple-boot Mac Book Pro helps :)
Take the material and deliver it. The concern here is is pacing and responsiveness. If a question is asked (even over Twitter), answers should be publicly provided for all to see. This is where you see, first hand, if you performed your analysis correctly. Are people getting it? Are they getting it too quickly? Are they asking really good questions on topics that you didn't think about?
And, the most crucial phase of all: evaluating your training. Surveys, verbal feedback, written feedback, and test scores paint a picture of the training provided. This allows for developers and instructors to improve upon their training. Was the length of training too long? Too short? Too complex? Too simple? Was the instruction professional or amateurish? One problem I saw with LIGATT's Twitter training was that legitimate questions were being asked in response to tweets, but were left unanswered. Very bad form, indeed.
But, that's not all...
The five stages of ADDIE lay out the process by which training is developed, and it provides for a solid foundation upon which a quality course can be built. But, it requires a bit more effort to get the content just right. In the Design phase we identified all of the objectives that are covered in the training. There's an additional step here: what is the take-away from each objective. This is referred to as a Knowledge Level, of which there are generally five*: (* some groups break it down into six, or seven for greater granularity)
1. Recognize - If you see a term show up later in life, you can recognize it and vaguely remember what it is. Flash cards. "Nmap, oh that's a security tool for hacking."
2. Recall - Upon recognizing a term, you can recall what the term is used for and the basics on how it works. "Nmap can port scan other computers across a network."
3. Comprehend - You can detail exactly WHY the objective is important. "To assess a target, I need to see if it has open ports and running services. Nmap is one of the few tools that can do this automatically. I could also use Nessus, but Nmap lets me ..."
4. Application - You know how to use the tool to perform a function. "I need to quickly assess a server, so I need to type `nmap -P0 -sV 10.5.7.2`".
5. Synthesis - You understand the objective and how it ties into everything around it, knowing when to best use it and when not to. "I need to assess this server but they have aggressive packet monitoring. However, their logs roll over every 8 hours. I need to slow down the Nmap scan to 9 hours between packets and netcat the results back to another machine for review."
If you would design a basic hacking course, you would end up with hundreds of objectives, each with it's own knowledge level. For keystone objectives, like an Nmap scan, or Metasploit attack, you would focus at the K4 level, at the least. For Metasploit, a K5 is virtually required as it requires an additional level of thought to understand the various exploits and payloads.
For less-used objectives, like using snmpwalk to assess SNMP servers, you would focus at the K3 or K2 level. And items requiring rote memorization, like port number assignments, would just be K1s.
K1s have their place, such as when memorizing a large set of data before a certification exam. Sales, Marketing, and Management staff generally work in the K1-K3 range. Your first level tech support works at K4, and the gurus work at the K5 level. Generally, if you are taking training to perform a critical job function, it should be predominantly taught at the K4-K5 level.
When all is said and done, you'll likely have months invested in properly creating a week-long training course. And people will scoff. Management scoffs. That's too much time! That's too expensive to develop! Is it, really? You build a strong, solid course that requires no oversight, is easy for instructors to pick up, has no ramp-up time for train the trainer, and requires bare minimum maintenance between iterations. Compare that to typical rushed courses that have major rewrites in between each iteration, and basically starts from scratch if the instructor quits and needs to be replaced.
Training is not just something you throw together. You don't just throw together a Power Point presentation to give a 16-hour course and call it done.
And so how is the LIGATT 15 Minute Hacker course failing? Because it is purely written and delivered at the K1 (Recognize) level. Vague attempts are made to give more in-depth details, but they completely lack context. There is no hands-on experience, no practical exercises, no testing, and no review process. It's the equivalent of yelling random sentences from a book to an audience in Times Square and calling it training.
LIGATT's How to Be a Hacker in 15 Minutes will not train you to be a hacker. Ever. You do not learn how to hack. You just learn basic terminology and phrases. At best, it will train you to be a script kiddie.
Then again, if you just want to throw together a quick, free, plagiarized training session just to try and drive your stock prices up... it may work out for you. Good luck with that.