CSITech - Computer Forensics

Forensic visualization Part 2 - Court Case

2011-12-17T16:08:00.000Z

Visualization gone serious

I blogged some weeks back on research I was doing around visualization of forensic data which was well received with some very interesting comments from readers (both of you!). However, the week after the posting I was asked to be involved in a prosecution of a man who was accused of various forms of grooming, sexual assault, voyeurism etc of several teenage girls in his community centre.

The case has now concluded and the man received 4 years prison, so a good result, however I wont name the case as I refer to the victims and they deserve as much anonymity as possible.

The case revolved around a large amount of Facebook chat between the accused and the girls, and between the girls themselves. Some of the chat was quite damning and on the face of it, it was clear that he was trying to talk the girls, one in particular, out of coming forward with what had been happening using emotional blackmail.

His defense on the Facebook chats was that the girls had logged in as him and had chats between themselves, implicating him in wrongdoing.

I was asked to consider the workings of Facebook, could they log in at the same time as him on a different computer, would he have a record on his own machine and what were the ‘relationships’ between the parties involved.

The word, relationships, got me thinking, could we visualize the data to ‘see’ the relationships and would it be easier for a jury to understand and interpret? Now, it is easy to map out Facebook ‘Friends’, the excellent Facebook Visualizer as well as the Facebook transform in Maltego will help with that task, but that doesn't really help us understand the activity that exists between those people. Although Im not much of a Facebook user I have load of buddies on Skype but some of them I haven't spoken to in years. Just because the accused and Girls A,B, and C were on each others Facebook lists and the fact that there was some chat doesn't ‘a relationship make’!

I used IEF 4(Internet Evidence Finder) to carve all the Facebook chats and fragments out of the 4 hard drives, it even did a great job on the accused’s Mac hard drive and I was left with 4 CSV files with thousands and thousands of chats. Now to make some sense of it.

I tidied up the CSV’s, removing some of the metadata that I didn't need and essentially just left the FROM, TO and the CHAT columns. Next I imported this data into Maltego as an Edge weighted graph. I expected this to cluster the chats around the person who made them and it worked better than expected.

Fig 1 shows the recovered chats on the accused’s computer and who he was talking to. Each orange dot is a person he has chatted with and the surrounding green dots are each individual chat. The primary cluster, centre left, is the accused with all his chats; being his machine we would expect this to be the largest cluster. As we can see there are many chats to many different people, however, our eye is quickly drawn to the 2nd largest cluster on the centre right. This is a person he talks to more than anyone. Rolling our mouse over the orange dot in the centre of the cluster, surprise, surprise, it is our 13 year old Girl B. The 3rd largest, at the bottom, is his best friend, but top right, Girl A.

Fig 1

This graph gives us an excellent tool, aside from just numbers and statistics as to who was important to him in a Facebook setting. The question, was this just a girl or girls with a crush, that it was one way traffic, is quashed by this graph, Girl B and Girl A are the 1st and 3rd most frequently communicated with persons on his extensive Facebook buddy list.

Encouraged by the success I did the same process on the machine of Girl B. This time, as there were many different chat partners I also removed the chats that only existed once or twice, the boy at school saying Hi, a friend inviting to a party etc, but which were not repeated with that person. The results in Fig 2 are fascinating:-

Fig 2

The primary cluster is of course Girl B herself, but no prize for guessing which cluster is the accused?? You’ve got it, the 1st next biggest cluster top left, in fact their chats are almost twice as many as any other person. Remember we are talking about a teenage girl here with lots of people to chat too and he was chatting with her more than twice as much as her best friends at school.

I then moved on to looking at the relationships with all those involved. I again used Maltego and imported all the chats from all the machines but removed the actual chat. This provided a link graph between the Girls and the accused and their friends, also showing connections between those friends. I will not present that graph as it includes the names of the persons involved but it showed the accused front and centre with chat connections with all the girls involved and showed the connections between those girls and their friends.

I felt this was very useful to a jury and so included it in my report to the prosecution barrister. It went on to form part of the jury pack so I can say that my graphs have made it to Court. Sadly, I was not called to give evidence on this occasion as the defense agreed all our findings and signed a statement to that effect. Shame really as I was looking forward to presenting this data in open Court and judging the reaction from a jury. Not that I am expecting wild applause and fist pumping whooping but it would be interesting all the same.

So far I’ve been using Maltego but have been given heads up of other free tools that might do the same job. The primary tool is Gephi, thanks @danmcquillan for the tip, a superb, free graphing application for Windows or Mac which supports many different output graphs. So far Im liking it, it takes a little more work pre-application as you need to define your Nodes and Edges for it to successfully graph the links. I’ve also had problems with the Preview and output elements which keep crashing, I need to pop a message on the forums really.

A Bump on the Node

Just for your information, the visualization industry seems to be dominated by research groups in Universities ‘visualizing’ everything that moves and then posting them on Youtube with no information about how it was done except the message ‘Arn’t we clever!’.

However, if you want to learn about it you appear to need the brain the size of planet, a doctorate in statistics and a student card. It is a very difficult area to start learning as a beginner. For example, search Google for - What are Nodes and Edges. Go on, try it. The top link is Wikipedia that presents you with a series of equations that make up graphing theory. Its a nightmare.

Anyway, for those of you out there with a shriveled 40-something brain like me, a Node is an element such as the person on my graphs and the Edges are the links between them.

Eg

I am Nick Furneaux.
My friends are Ed, Toby and Chris
I talk to Ed and Toby
I never talk to Chris

The Nodes are:-

Nick
Ed
Toby
Chris

The Edges are:-

Nick - Ed
Nick -Toby

The graph would show links between me and Ed and Toby but Chris would be an unlinked orphan node floating around the graph on his own. Sorry Chris.

Clear? Good.

Hear endeth the lesson!

Evidence visualisation

2011-10-26T14:35:00.002+01:00

I've been doing a load of research on trying to easily visualize digital forensic data with the hope that patterns, frequencies and clusters would stand out easily. There are already excellent tools that do a great job for primarily email such as NUIX and Intella, but these are pretty expensive beasts. You can also look at software such as I2's Analyst Notebook but now we are talking stratospheric money, out of my league.

My mind was focused when a friend at the Met Police introduced me to a new tool call Bulk Extractor from Simson Garfinkle which scans across an image and extracts data strings, very quickly, based on a plugin structure. I set out to run Bulk Extractor against a RAM image and had tremendous results. The tool will extract email addresses, URL's, search terms, Credit card numbers, telephone numbers and others, and does so with aplomb. The tool generates a list of text files which can be analyzed with the Bulk Extractor Viewer. You can run it against disk images, phone memory dumps and RAM. This is great, but when faced with a list of 10,000+ URLS where do you start. This is where some visualisation help really comes in.

After alot of looking around I came back to a tool I have used many times, Maltego. Maltego is primarily used for the enumeration of Internet data, connecting IP's, WHOIS, email and domain information to enable the mapping of an online infrastructure. It also enables the importing and graphing of text/csv files.

I ran Bulk Extractor against an old 512meg RAM dump and amongst other things it extracted URL links between over 3000 IP addresses. Normally I would move on quietly(!), however, I tidied up the columns in Excel and imported into Maltego, mapping the URL address columns. This is what I saw:-

Each little cluster represents URL's linking to a central URL in the hub. A quick look shows the most popular URL's at the top with many links. Straight away the list of 3,000 is somewhat more manageable if we are interested in popular links.

Zooming down we see:-

Although a tad tricky to see there are little links between the nodes with URL addresses linking to the primary URL. We simply draw around a cluster and then we see:-

Although the URLS linking in are hard to see, believe me they are there, showing all the URLS that link to the central Mozilla.org URL. How cool is that?

Next I thought IP addresses would be fun, except we had over 10000 entries from the one RAM dump. However, it mapped very well:-

Again there are some very obvious clusters which may be of interest. Scrolling in we see a very definite structure:-

Scrolling in further we see all the interconnected IP's with a very interesting structure with clusters grouped together into super-clusters.

Further again and we see the individual addresses:-

Now we can see each individual connected IP and their port numbers. Now Maltego really comes into its own. We select the centre of the cluster and select the Transform to reverse look up the domain and TLD. As if by magic the graph redraws this cluster and we get:-

We now can see that all of these IP's are referencing back to Yahoo.com and it is a very popular cluster in the RAM dump.

Being able to 'see' data in this way can help the investigator to quickly zone in on the important areas, seeing, if you like, the wood for the trees.

I'm now doing work on mapping outputs from Volatility and will blog again in a few days.

Cheers

Nick Furneaux

Downloading files on your iPhone

2011-09-14T10:02:00.000+01:00

I just cannot believe how long its been since a blog post, there are just not enough hours in a day. Then, when I do pop a post up its nothing to do with forensics, great!

I wondered if you have ever had the issue of browsing on your iPhone when you find just the file you are looking for, perhaps a tar, zip, dmg or some other file type that the iPhone does not let you download but that you don't want to browse away from and risk losing for good. I've found a simple way to achieve it.

If you download the Dropbox app it becomes a option to 'Open with' when browsing the web. Simply:-

1. Browse to the file you want to download

2. Select Open in Dropbox from the screen and it will copy the file from the site to your Dropbox box account letting you access it from your computer later.

Its already proving to be very handy indeed. Give it a go.

One other small thing, if you hold down shift on your Mac whilst minimising or maximising a window it does it in cool slowmo! Who knew!

Intel SSD's have default AES encryption - worried?

2011-03-31T22:23:00.004+01:00

Intel have announced their range of new SSD's with a range of security and data stability tools, the 320 range. The include sizes from 40gig to 600gig (if you have the money!) and my experience is that they are crazy fast. Putting your OS on one of these would make a huge difference to the speed of the overall machine.

However, Intel state that they come with a default AES 128 full disk encryption system which apparently successfully finds the trade off of speed and encryption/decryption. The thought of new machines coming already set up with an AES flavour is enough to make the average digital investigator hang up his mouse and go stack shelves in Salisbury's (small print - other supermarkets also offer shelf stacking opportunities) . Should we be worried?

No.

It is true that the disk, out of the box comes running a AES 128 key providing full disk encryption. However, plug the disk into your machine and it will run with no seeming encryption involved at all? How so? Simply because there is no user key set up as default. To make the encryption 'work' as a security layer the user has to set up an ATA BIOS user password to secure the encryption key. Don't set up a BIOS password, no useful encryption. Excellent!

You can check out the security document here.

Knowing bad guys, and most of us have the misfortune of knowing their computers rather well, they are notoriously mistrusting of encryption and it is unlikely that the computer they buy will come with a big sticker saying how vital it is that they set a BIOS password. Indeed, many people believing that they are experts will read the drive specs, see AES 128 and believe that they are more secure than NASA. All which makes me think I should delete this blog post? Ah well, no one reads it!

Exif and GPS data on a Mac

2011-03-04T10:11:00.004Z

I was kicking around yesterday looking for a decent Exif viewer for the Mac, I found one or two but they didnt support extraction of GPS data. Turns out my time was wasted and OSX supports and reports Exif data including GPS location data.

Step 1. Open your image in Preview mode.

Step 2. Cmd-i to Open Inspector

Step 3. Click the 'i' tab and select Exif or GPS button

It even has a 'Locate' button to fire the coordinates up in Google maps. Simple and brilliant.

Although there isn't an export feature, the dialogue does allow you to copy and paste the data out into a text program.

Gotta love your Mac!

Volatility 1.4

2011-02-16T22:23:00.002Z

This is just an initial post about the beta availability of Volatility 1.4. I've been teaching 1.3 as part of my Advanced Live Forensics course for 18 months or so but it only supports XP SP2 and 3 RAM images. The new 1.4 version from the devs and helpers at www.volatilesystems.com have been toiling over this version for somewhile and its great to at last have a play with it.

First things first you can find proper 'how to' resources at http://code.google.com/p/volatility/ but downloads are currently limited to within svn. If this is new to you its easy enough. If you are using a Mac with Snow Leopard just open a terminal and type 'svn checkout http://volatility.googlecode.com/svn/branches/Volatility-1.4_rc1'. This will download the 1.4 version and put the Volatility files in your user root folder.

Once downloaded just 'cd Volatility-1.4_rc1'. Anyone used to the old version will see a small difference in the running of the commands. Instead of-

python volatility pslist -f [pathtoRAM]

..you have quite a different syntax. It breaks down like this-

python vol.py [plugin] --profile=[PROFILE] -f [image]

vol.py replaces the old volatility framework command
plugin is the command such as pslist, psscan2 etc
profile is completely new but a vital component of the new framework. For all RAM images except from Windows XPSP2 x86 should have the profile defined at the --profile switch. The BasicUsage document lists them as:-

PROFILES
--------
VistaSP0x86 - A Profile for Windows Vista SP0 x86
VistaSP1x86 - A Profile for Windows Vista SP1 x86
VistaSP2x86 - A Profile for Windows Vista SP2 x86
Win2K8SP1x86 - A Profile for Windows 2008 SP1 x86
Win2K8SP2x86 - A Profile for Windows 2008 SP2 x86
Win7SP0x86 - A Profile for Windows 7 SP0 x86
WinXPSP2x86 - A Profile for Windows XP SP2
WinXPSP3x86 - A Profile for windows XP SP3

So running a basic pslist against myram.dd imaged from a Windows SP3 box would look like this-

python vol.py pslist --profile WinXPSP3x86 -f myram.dd

In the previous version outputing the results to a file could be achieved by using '>' or '>>' to output to a text file etc such as -

python volatility pslist -f myram.dd >> pslist.txt

However, in 1.4 we have many more options, by adding -

--output= you can specify numerous output types if the module being invoked supports it. This includes -

--output=text
--output=html
--output=csv

To check what a module/plugin supports just check help - python vol.py pslist --h and look for the output section.

You can add -

--output-file=myoutputfile.csv to name your output file. So our previous command line could look like this -

python vol.py pslist --profile WinXPSP3x86 -f myram.dd --output=text --output-file=myfile.txt

That should get you started.

There are also some exciting new modules to play with such as bioskbd a plugin based on Andreas Schusters work. It enables the reading of input text from the BIOS area of memory which can include the BIOS password or even Full Disk Encryption passwords. Check out the link to Andreas site for more information. This plug in has apparently been around for a while but I'd completely missed it. If you do check it out take note that some RAM dumping tools dont image that area of RAM. For example if you are using Matthieu Suiches win32dd tool you need to add '-t 1' to grab page zero.

Also there are some exciting malware analysis plugins such as svcscan which can list Windows services from both usermode and kernelmode and also ldrmodules for detecting unlinked DLL's.

Anyway, thats all for now, I'll try and post more in due course once I've had a proper play.

Nick

Mac Ram Dumps

2011-01-20T14:24:00.004Z

Well its finally happened, at last a tool to dump RAM from OSX. Big thanks to ATC-NY for their Mac Memory Reader which can be downloaded for free here.

The tool is very easy to use, simply unpack and open a terminal.

cd to the folder MacMemoryReader (For newbies something like - cd /Users/name/Desktop/MacMemoryReader

Run - sudo ./MacMemoryReader filename

..where the 'filename' is the path to a connected storage device

You will prompted for your admin password and off it will go.

Remember to check that your connected storage has enough space for the entire RAM dump.

If you want to feel part of the action you can throw a -g into the command line and it will provide a percentage notifier.

The program outputs a Mach-0 raw file which should respond well to data carvers and the like. Well I've only conducted a couple of tests but Photorec and Foremost do a cracking job of getting at the files. They both successfully retrieved HTML, jpg, zips and a whole variety of other files including web pages going back 3 months. My 8 Gig of Ram offered up over 38000 files. Many of them were fairly uninteresting txt files so you need to wade through to find the good stuff.

If you are trying Foremost just bear in mind the 3Gig limit, perhaps take a look at Scalpel.

The next step is to start looking for running process information, fairly critical in basic RAM analysis. I'm away teaching next week so will have some evening time to play.

I'll try and blog again soon

I Won Something!

2010-07-20T14:36:00.003+01:00

I've never been big on entering competitions, mostly because maths gets in the way. You do a quick calculation on the odds of winning anything of note and realise your time is better spent working to actually earn some money the old-fashioned way.

It was rather a surprise to learn that I'd been shortlisted on the Forensic4Cast awards as Digital Investigator of the Year. It was even more surprising to win it! I would have loved to have been in Washington for the award ceremony but there we go.

Anyway, thanks to Forensic4Cast and everyone that voted for me, I'm over the moon, and looking forward to getting the award.

Thanks also to my makeup artist, my parents for all their hard work and Yoda for sticking with me throughout my Jedi training. I may cry.

Im Famous, or infamous, or neither.

2010-06-18T18:07:00.002+01:00

Short blog this time with some shameless electioneering. I've been shortlisted as Digital Forensic Investigator of the Year.

Visit http://forensic4cast.com/2010/06/16/forensic-4cast-awards-2010-voting-is-open/ to vote. Doesn't have to be for me of course!

OSX RAM Acquisition

2010-05-12T15:46:00.007+01:00

Acquisition of OS X RAM is a bit of a holy grail of memory analysis, quite simply because no-one has done it, or has admitted to it. It is always good form to realize that whatever we think of as secure has probably been undermined by Dark Forces working from bunkers under grassy fields, or desert, or tundra depending on your Government Agency of choice.

In Leopard there were some significant weaknesses in OS X RAM, well researched and documented by Dai Zovi (We're not worthy!) who demonstrated in 2009 a number of different attacks on the OS through the poorly implemented memory stack which enabled heap allocated memory to be executable, unlike Vista/7 etc - Windows more secure - who knew!!

Snow Leopard with its 64bit architecture has gone a long way to solve that. But with the incredible amount of information available from a Windows RAM dump it would be great to achieve the same from a Mac. Work has been done with DMA (Direct Memory Access) via Firewire which can theoretically work and some researchers had some success with Leopard but its all gone quiet with Snow Leopard. So where does that leave us?

Well, unless you are prepared to freeze the chips you need to acquire the RAM whilst the machine is live. On a Linux machine you can simply dd /dev/mem and /dev/kmem but no such luck with OS X.

For the time being our best bet is the OS X counterpart of hiberfil.sys. In Windows hiberfil is a file generated in the root of C when the PC is put into hibernate state. The resulting file can be converted into a raw RAM dump using either tools from Matthieu Suiche with the Sandman project or the version produced for Volatility. OS X has a similar file called sleepimage. You can see if your Mac has one at the moment by doing the following:-

Open terminal
Type - cd /var/vm
Type - ls

If your machine has been hibernated you should see a sleepimage file with a file size that is the same as your RAM.

If you come up against a running Mac and will be seizing it then it is possible to force the machine to create the sleepimage file.

Suggested 'Forensic' methodology:-

Open Terminal
Type - sudo pmset –a hibernatemode 1

When you shut the lid it now creates a hibernate file and shuts machine down rather than putting it into sleep mode. The problem is that it will likely ask for the admin password. You could run MacLockpick which will extract the Keychain and possibly give you the password you need.

Next, you need to set it back - sudo pmset –a hibernatemode 3

Shut the lid, take the machine.

Now simply image the drive as normal and extract the sleepimage file and analyze.

If you were doing a live data acquisition or search of the machine it is simply the case of plugging in a USB drive and typing:-

sudo cp /var/vm/sleepimage /Volumes/USBkey (Where USBKey is the name of your drive.)

Now the problems:-

Changing the hibernatemode makes a technical change to the machine.
The technique forces you to shut the machine down which is no good if you want the RAM live whilst leaving the machine running.
There are currently no tools available for the analysis of the sleepimage. The tools we use for Windows RAM analysis such as Volatility, Foremost, Memoryze etc do not work. Get coding!

This post is not desperately useful as it just explains how to get a pseudo-Ram dump out, what you then do with it is up to you. If you figure anything out I'd love to hear about it!

IPad, the non iPhone and non net book

2010-04-12T22:09:00.003+01:00

Well I'm doing what thousands of bloggers will do in the next few weeks and writing a post about their shiny new iPad whilst writing it on said device. And here it is. An iPad. It's thin, fairly weighty and I feel like a very small person in a Lilliputian universe typing on an iPhone.

The iPad box arrived via DHL from the US the day after release and the family sat down for the social and yet rather sensual task of unwrapping an Apple product. The top slid off with a satisfying whooshing sound, possibly in my mind, and there it was, covered by the familiar cellophane wrapping, a big iPhone. I unwrapped it and held the big iPhone in my hands. It felt like it wanted to be dropped, slim and too slippy until I discovered the Apple sign on back in more grippy material which just a finger on makes it feel more secure.

Plug it into the Mac and turn it on. No iPhone\iPod clone here. Oh yes it is, just bigger icons. ooh and look you can swish your finger from page to page just like....umm my iPhone. First job, connect wifi, no issues here, straight on. Open Safari, key news.BBC.co.uk and .... Oh my goodness it looks fantastic. I spent the next half hour just browsing the web, especially news sites. No question, this is the best way to browse the web. It is so natural, so like holding a book, just sit on the sofa and read, sweeping between sites with ease. Sorry, if you wanted to hate the iPad, then never try browsing BBC news or The Times. It is just awesome.

Next I downloaded several new apps, the Epicurious recipe app, which is fantastic, the new accuweather app, beautiful, Real Racing HD for my son which is brilliant. I have to apologize but I just love this device.

Now seriously what "is it"? Is it a net book with no keyboard or a big iPhone? Simply neither. This is a new device, a perfect form factor for reviewing and browsing data. For producing data it is honestly a bit rubbish, the keyboard is ok and I can now type pretty fast but it's no replacement for a proper keyboard. I think I would happily write a few emails and if stuck on a plane with no laptop battery life I would write another blog post but it wouldn't be my first choice. However the last paragraph was written without editing or deleting mistakes and I think it's all ok.

Now what about battery life, Apple say 10 hours. I first charged this Thursday of last week in the evening, it got a pretty heavy hammering by the whole family including games and lots of browsing and kindle style book reading. It didn't go back on charge until Sunday evening which I think is pretty blooming brilliant. It's been off charge all day and been in use constantly for the past 3 hours and the battery life still shows 66%. Not bad. The battery got a real hammering at my local Apple store today, none of the guys there had seen an iPad and wanted me to pop in with it. It was interesting to see them having their photos taken with it, star status!

The other app I have is Air Sharing, this let's me set the iPad as a hard drive on my Mac. I can drag and drop files onto the iPad and review them on the go, very easy. I tend to carry a lot of research stuff, PDFs etc so this will be excellent. The reading size is perfect and with no boot time you can be reading your document in 5 secs.

I'm flying to Hong Kong next week and this will be my device of choice on the plane, I can read a book, very clear actually, watch a film, superb screen quality and play a few games, what else do you need sat still for 14 hours? Yes it is just a big iPhone but the form factor makes it a superb device, not a laptop, not an iPhone, it's an iPad.

Skypeex - additional comments

2010-03-11T15:41:00.002Z

I've had some very good feedback about the Skypeex tool and I appreciate all your comments.

One or two have not really seen the point of the tool as there are plenty of Skype log viewers around such as from Nirsoft and Skypr. I will repeat what I posted on the LinkedIn discussion board.

"the Nirsoft tool, and others, are log viewers and this presupposes that you have access to the disk/logs. A covert live acquisition will often just take RAM and other volatile data, RAM may be taken before the plug is pulled only to discover that the disk is Full Disk Encrypted or that the logs are in a Truecrypt container. The user could even be using 'Portable Apps' Skype on a USB key which would mean no log files at all on the disk, however the data could still be in RAM.

This little tool is not meant to be a replacement for the excellent chat log viewers out there but provides a way of getting the data from RAM where circumstances dictate."

I'm working on an improved version where Strings isnt needed and hope to have that sorted in the next couple of weeks.

Skype Chat Carver from RAM - Skypeex

2010-03-09T17:33:00.008Z

Well I was going to keep testing but it just seems to keep working, so here it is in version 0.5.

Download

I’ve been teaching my RAM analysis course for about a year now and enjoy working with Volatility and some other open source tools. I’ve been making use of Jeff Bryners cool little Python script (http://www.jeffbryner.com/code/pdgmail) to extract Gmail artifacts and was motivated to do the same for Skype chat and any other Skype stuff that might be hanging around in a RAM dump.

The only problem was that, although I’ve done a bit of programming in the past, Python was a long hissy thing you wouldn’t want to meet on a dark night. Having gone through the pain of programming ‘Hello, world’, simple Pokemon text games for my lad and tedious maths exercises, I’ve actually managed to produce something meaningful.

The idea is to extract Skype chat lines with their associated meta-data, which includes timestamps, the Skype names in the conversation and the author etc.

The complete Skype line in RAM starts with the magic value ‘INTO Messages” followed by column headers then the values of the chat line including the chat body.

This is very much work in progress but will simply do the following:-

1. Run Strings against your RAM dump
2. Run the Skypeex tool against the resulting Strings file
3. It will carve out all the Skype chat lines it can see as well as trying to find and extract all the Skype sessions and ‘orphan’ chats that have been created.

It’s interesting to note that the latter process even seems to find the ‘spam’ message sessions that you sometimes receive.

This has been tested on dump files from Windows XP2 and XP3 with Skype 3.8 through 4.2.
I don't currently have a Windows 7 box up and running, if anyone has one available please let me know.

Please do not hesitate to get in touch with ideas and improvements.

Usage:

There are 2 versions in the zip file.

skypeex.py is designed for use under Python 3.1.1 and above

skypeex26 is designed for use under Python 2.6

Due to changes with several commands between 2.6 and 3 they are not interchangeable, although the differences in this code are only in the input and print lines.
For best testing results, have several Skype IM chats with friends and then image your RAM. On a windows box, use any tool to grab RAM (tested on Win XP SP2/3):

I recommend Win32dd (or Win64dd) from Matthieu Suiche - http://windd.msuiche.net/

Run strings against the RAM image (e.g. Windows version can be found in Helix distro)
example: strings c:\ramdump.dd > c:\stringsout.txt

On linux box do:
strings ramdump.dd > stringsout.txt

Script usage -
from command shell - python skypeex.py - then, when prompted, simply provide the path to the strings output file.

The output files will be written to the folder where the script is run from. The output is a CSV file with chats (incl headers) and a txt file with extracted skype sessions and carved orphan chats. Please expect many duplicates and some false positives.

In the CSV file the 'Timestamp' column is the date and time of the message in UNIX time. Sorting on this column gives you a timeline of messages. I'm writing a UNIX time decoder but it doesn't work yet.

The primary message content is in the 'body_xml' column.

Code:

The key elements of the code are:-

if "INTO Messages" in line:
def extract(text, sub1):
return text.split(sub1)[-1]
str2 = extract(line, 'VALUES (')

This searches for the magic value, strips out the rubbish and returns the comma delimited values we are interested in. This includes:-

Chatname – the initiator and recipient of the session
Timestamp – The time and data the message was sent in UNIX time
Author – the sender of the message
From_dispname – the screen name being used by the sender
Body_xml – the body of the message, can slip into the chat_msg column
GUID – session identifier

Next:

if "#" and "/$" in line:
outfile.write(line)
nxt = next(data)
outfile.write(nxt)

This time we look for the existence of the # and /$ characters in the same line. This refers to the pattern written to RAM of each Skype session, which looks like this:

#nfurneaux/$bennyboy1982;810b0fd9ef04db08

This shows the 2 persons in the Skype session with the first name being the initiator of the conversation. I’m still trying to figure out the hex value at the end, but it seems to be a GUID session number, any ideas let me know.

Sometimes we recover session line like the following:

#bennyboy/$nfurneaux;9fa7c85b71354392Jd1bbennyboy1982Ben Brown
#andyw/$nfurneaux;9fa7c85b71354392Jd1TnfurneauxNick Furneaux

We are able to see the actual Skype name as well as the screen name being used during the session. The cool thing is that we also grab the next line with often includes actual chat associated with the recovered session. Hence we capture:-

#bennyboy/$nfurneaux;8f915423c984767aJ[VonfurneauxNick Furneaux
ok quite close
# bennyboy /$nfurneaux;8f915423c984767aJ[bennyboy Ben Brown
Aug 23
# bennyboy /$nfurneaux;8f915423c984767aJ[VQnfurneauxNick Furneaux
when are you presenting at HTCIA
# bennyboy /$nfurneaux;8f915423c984767aJ[bennyboy Ben Brown

Interestingly this conversation is carved in reverse. We can ascertain that bennyboy started the conversation but see the sender in the second part of the session line, followed by the chat.

I've never released a tool to the community before so be kind! Let me know how you get on.
Nick Furneaux

Download

Unfit and unblogged!

2010-03-09T17:23:00.002Z

Im just preparing to release a Skype RAM carver written in Python and I thought that my blog would be the best place to put it. However, I just checked it to make sure I remembered how to log in and noticed that my last blog was in Oct. This is a coincidence as Oct was the last time I went for a run! I was thinking that there was no correlation but actually, moving house, traveling all over the place and a very busy work 6 months has contributed to both.

Yesterday I went out with my lad and ran for 2.5 miles, including loads of up hill and was pretty surprised at my retained fitness, which is good, however my blogging looks in much worse shape.

It doesn't help that the eponymous Happy Monkey is regularly blogging fabulously funny and insightful ditties that anything I do will be put to shame. However, watch this space for a free, and rather cool, Skype Chat RAM Carver.

2009-10-23T14:53:00.001+01:00

Although I quite like this blogging lark, you will notice from the total lack of activity in recent months that I’m not very good at it. Fact of the matter is that I’ve been extremely busy, which I guess in the current climate I should be thankful for. Computer Forensics is a good career choice in a recession as, simply put, there are always bad people. In fact there is some evidence that white collar crime (and today that almost always involves computers) is on the rise as people worry about jobs, mortgages etc and when an opportunity to pilfer away a quick buck is found, many will succumb.

I’m writing this on a train to the Midlands where I’m helping a Bank improve its analysis of Malware written specifically to target its customers. This too is on the rise with phishing attacks commonplace. The problem with Malware written specifically for a task is that the AV products often don’t have a signature for it and hence it renders itself fairly invisible even from the ‘Heuristic’ scanners. To counter this it seems that the AV companies are lowering the bar, almost every time I write a script or compile a new piece of code, Kaspersky or AVG or McAfee scream that its Root Ware, or a Trojan or something equally nasty.

If you download virtually any of the fabulously useful tools from Nirsoft (www.nirsoft.com) such as their password recovery, USB key parser or Wifi tool and, wham, ‘It’s a virus!!’. No its not. Cain and Able password recovery tool recently started triggering an alert, Nessus fires an alert….what is going on. It feels at the moment that any software tool not in their database is automatically a Trojan come to steal your car, wallet and way of life.

Anyhow, rant aside, there has to be a better way of analysing Malware and I think RAM is the answer. Nothing can hide in RAM, processes hidden from the OS can be uncovered in RAM. Many tools do a process called List Walking to discover processes running in live RAM or a RAM dump however manipulating the DKOM object can render a process out of the ‘flow’ and essentially invisible from the OS or from list walking programs. Psscan2 in the volatility suite overcomes this by scanning the dump file for process objects whether or not they are connected to others. Outputting this view in a dot format and opening in something like graphwiz provides a fantastic, clean view of the running processes and their threads. Simply invoke by:-

Python volatility psscan2 –d –f > output.dot

Analysing the process start times, thread and parents, exe path and other variables provides a very ‘quick win’ when searching for malware of any type.

This is a manual process and would be tricky to automate but very worthwhile to do if malware analysis is your business.

Few minutes til the train is due in so will speak later, hopefully sooner!

ACPO and RAM Analysis course

2009-07-09T09:39:00.003+01:00

It’s been a busy few weeks which is why I haven’t had a chance to blog for a while. I had the opportunity to present at the ACPO Conference 2 weeks back which is always a good event, with friends and colleagues from many different Forces and Agencies. It is normally a chance for a late night drink but exhaustion from the past few weeks activities had me in bed by 11pm each night.

My brothers company, Bright Forensics, was exhibiting there and had e-fense’s Eric Smith on the stand. Eric is a very talented investigator and has a tremendous knowledge of the forensic world and marketplace. They were focusing on touting e-fenses’ Live Response key. This is a USB key designed for fast and easy acquisition of live and volatile data from a running machine. In my view it is the first tool that provides an ease of use capable of being used by a front line arresting officer. I know that this is a sensitive subject at the moment, but a plug and play device that will grab Internet History, RAM and other useful data, is a very interesting addition to an officers arsenal. Discuss ☺.

The buzz word of the conference was ‘Triage’. In simple terms the phrase is being used to suggest that we could use a device or software tool to ‘search’ a machine and include or exclude it from an investigation, hence shortening backlogs that exist in most HiTech Crime units . Umm. I have a real problem with the idea of triage in this situation. In a hospital or emergency setting triage is used to prioritise not exclude and I think this is where such tools could have a role. If you get 5 machines in for a CP case, prioritising the machines, perhaps quickly locating the one with the primary evidence could work fine. However, I think that we will struggle to never image or investigate those other drives. If I think as a defence expert I may suggest that although there was a large amount of evidence on one drive, evidence existing on the ‘sons’ or ‘lodgers’ computer could lend credence to the fact that someone else used the computer belonging to the accused. I appreciate this is somewhat simplistic and perhaps the initial data might make the chap stick his hand up, but I’m sure that you can still see my concern.

Last week I taught my first Advanced Live Forensics course with a particular focus on RAM analysis. I don’t mean to blow my own trumpet but I think it was a resounding success. A chap from one of the UK Counter-Terrorism units suggested that it should be required learning for all computer forensic people and another was impressed by what he called the ‘first new computer forensic discipline since the advent of disk forensics’. Overall, I was chuffed. Obviously this is rapidly turning into an advertisement which I apologise for but if you would like to come then you can find dates on the www.csitraining.co.uk website!

ACPO and RAM Analysis course

2009-07-09T09:39:00.001+01:00

Imaging Windows 7 Live

2009-06-05T14:34:00.003+01:00

I've been spending some time working with Matt Blackband today on issues surrounding imaging Windows 7 disks and RAM. I've got a copy of Windows 7 32bit RC1 installed under VM Fusion with 2 Processors and 2 Gig of RAM allotted to it.

Before I start I just want to point out that although I have quite a bit to do with e-fense on a day to day basis including teaching the use of Helix 2.0, I do not make anything out of the new Helix Pro. This bit of research was just myself and Matt wanting to see whether it worked well under Windows 7 and compared to Helix 2.0. This is NOT an infomercial!

Although there has been alot of talk about exFAT and its uses, Windows 7 installs with NTFS as default and installed very quickly indeed. There have been some concerns and questions over whether our current typical live forensic tools would be able to successfully run and acquire drives and RAM. As Helix is a personal favourite tool and one that I teach, I focused my attention on that.

I loaded the latest Beta 2 version of Helix Pro (Should be released soon) which loaded quickly and successfully. Helix Pro saw the connected drives and partitions and also correctly reported the RAM size. Running the Helix RAM acquisition I was able to acquire 2 Gig of RAM, writing to a shared drive on the host MAC in a little over 2 minutes which is very good indeed. I was then able to successfully run Strings and Foremost to extract text data and carve files respectively. As expected Volatility refused to run and we wait to see if a Vista/7 update is forthcoming?

Disk imaging also worked correctly as expected for making both a RAW and an Encase 6 image, also creating disk and imaging information and checksum PDF's.

One of my favourite aspects of Helix Pro is its lightening fast volatile data acquisition. I was a little dubious that it would work under 7, but work it did, finishing in less than 20 secs and producing a 96 page report! Enjoy reading that!

Helix 2.0, the remaining free offering, as expected, did not fare as well. The GUI fires up OK but you are unable to trigger a command shell from the GUI as no Windows 7 shell exists on the disk, however browsing to /IR/Vista, and opening a Vista cmd file directly and then running cmdenv, did provide a usable shell which enabled me to run binaries on the disk.

System Information worked correctly reporting Owner, Network and Logical disks.

As expected the GUI would not image RAM or Disks although extracting MDD from /IR/RAM to a USB key and running it, successfully imaged the RAM in a little under a minute to the local disk (not recommended in the real world :)).

After some down and dirty testing today it is good to see that Helix Pro is up to the task of working with 7 which I guess makes it a £200 tool worth having in your toolkit. Of course, it will be interesting to see the take up of 7 after the lack-lustre reaction to Vista, but I have to say, even as a hard and fast Mac user, its not too bad. It installed very quickly and just worked out of the box. The interface is clean and simple and programs pop up nice a fast. Could this be a 'good' version of Windows? Time will tell. More research to be done.

Apple and Pears

2009-04-24T15:42:00.002+01:00

I had a couple of chaps turn up for a meeting yesterday from a certain UK Law Enforcement Agency and due to a crackingly sunny day were able to sit in a pub garden for a late and leisurely lunch. Anyway, thats not the point!

One of the chaps, Simon, pulled a little Netbook PC out of his bag and low and behold it was running OSX. It was really impressive to see such a tiny machine, designed for Linux or Windows to be running, very successfully, OSX in all its 'never crashing' glory. Being very small and light its essentially a MacBook Air but about £1000 cheaper.

I guess because I'd never gone to look, I did not know that since Apples move to Intel chipsets there has been a huge amount of effort in the hacking community (I use the word hacking in its proper sense) to get OSX successfully working on PC architecture. The Netbooks with their Intel Atom processors are, apparently, perfect.

Wired magazine wrote about it late last year (http://blog.wired.com/gadgets/2008/10/os-x-running-on.html) with similar results, although they noted that some elements such as Wifi and Sound fail to work on some Netbooks including the one they tried.

A very good list of Netbooks with the elements that work or do not can be found at http://gadgets.boingboing.net/2008/12/17/osx-netbook-compatib.html. It appears that the Dell Mini 9 is perfect and virtually anything can be made to work.

It is worth noting that although a great fun project, by loading OSX onto a 3rd party piece of hardware you are breaking the Apple licensing agreement, really fancy getting a Dell Mini on order though :)

...and the Supercomputer gets even better!

2009-04-17T12:00:00.002+01:00

Since the Supercomputer got fixed I've been doing some tinkering with quite staggering results. Elcomsoft have released a new version of their Wireless Cracking tool and you can now specify multiple dictionaries which is very useful. In addition, ATI now have new drivers that improve the GPU acceleration so I've got those downloaded and installed.

It then occurred to me that processing time would be taken up with the software figuring out all the permutations for each word in the dictionary, so I took a good 3 million word dictionary and ran it through the permutation generator that is part of John the Ripper.

john -w:dictionary.txt -rules -session:johnrestore.dat -stdout:63 > newdict.txt

This turned a 40 meg dictionary file into a 1.6 Gig monster with a staggering array of derivatives for each word. Feeding this into the cracker I have now raised my cracking speed from around 18000 passwords a second to a mind-blowing 45000 per second, or 3.8 billion a day. Not too shabby!

To deal with purely numeric WPA passwords I've got a friend writing a bit to code to generate a dictionary with every permutation up to 10 billion which is a nice long 11 digit password. Although we are looking at the best part of a week to run I believe that it is worth the effort.

Crack on - if you pardon the pun!

Expoliting the MSN protocol

2009-04-16T12:01:00.002+01:00

This is a post where I am not going to say anything :) I'm not going to say what we have found, what we can do and how we do it, but let me explain the problem.

Many Police Agencies have an interest in where a particular Internet user may be located and to achieve this, detecting their IP address and then asking the ISP for user information is a great way to do it. It is no secret that some Agencies monitor chat rooms and ingratiate themselves with known offenders on Instant Messaging (CEOPS invited the BBC in last year to discuss this), however chat using something like Windows Live Messenger proxies and anonymizes at Microsoft meaning a whole load of paperwork is needed to get the actual subjects IP.

Well that's the problem and Microsoft say that there is no way to circumvent this issue. If you are in this position and would like to discuss the 'problem', you know where to find me.

...and it breaks

2009-04-16T11:54:00.002+01:00

In addition to my last post, after just a couple of days of password cracking my super-beasty computer packed in. It seems the 4 uber GPU units decided to up and die which is not helpful when everything is GPU accelerated. Engineers turned up and we are firing on all cylinders again.

Interestingly I am now getting the full 20,000 passwords per second cracking speed that I was expecting whereas before I was only getting a fraction of that, I think there was something wrong from the start. As I look to my left a cracking job for a Police Agency is running at 18,000 per second, not too shabby.

'Super' Computing!

2009-03-03T21:56:00.004Z

It's been a big day! My supercomputer arrived in a rather large box, much to the obvious annoyance of the delivery man who had to drag the thing 30 yards as he couldn't get the van up the lane near the office.

Unwrapped and connected up to a suitably large screen the beasty purred into life and promptly crashed. No Apple technology here. Side off, found a couple of loose cards,tighten up, reboot and we are away.

The machine is based on AMD motherboard technology with 2 uber ATI 4870X2 boards providing 800 parallel processing cores per board giving a total of 1600 processing cores. With the right software designed for GPU parallel processing it will chug along at 2.4 terraflops or 2.4 trillion floating point calculations per second.

The definition of a supercomputer is 1 trillion terraflops and the first one was built by Intel just 11 years ago, it took up 2000 sq ft of space. 11 years on I have a machine 2 1/2 times more powerful under my desk, the lights dim when I fire it up but you can't have everything!

I've bought it to carry out super fast password cracking, I can chew through 60,000 passwords per second or 5.1 billion per day which is some work rate especially when using intelligent varying dictionary based attacks. Instead of pure brute forcing which is all down to key space (password length * all possible combinations), an intelligent varying dictionary attack takes a word such as 'password' and attempts all likely variations such as :-

pa55word
pa55wOrd
6a55w0rd
password1
password123 etc etc

Using this process a 3 million word dictionary can quickly be turned into a 150 million word table or much more. When done 60,000 times per second you can try an awful lot of variations and the success rate becomes very high indeed. A completely, pattern free, randomized password/phrase will still require brute forcing and we will all probably retire before a guaranteed success.

The new software I'm using focuses on WPA 4 way handshake attacks, you can check it out here. Other software allows the GPU accelerated attacks against Office files and loads of others.

My first job arrived from a Police Department yesterday so we shall see how it goes.

We're jamming

2009-02-11T22:50:00.002Z

Again, I have been neglecting my blog and I apologise! Little one was in hospital for the first 2 weeks of the year and I've taken 3 weeks to catch up.

Continuing with the wireless attack theme I came across an interesting way (illegal) to force a deauth. As some of you will know, to get the 4-way handshake needed for WPA cracking you need to force a authentication of a client and pick up the transaction of packets as it reauthenticates. However, this is easier said than done and does not always work.

One way to ensure deauth is to employ a hardware wifi jammer. You can readily source from the Far East a jammer with 30+ft range which is sufficient to take out a house's wifi network whilst walking by. Now I am at pains to say that jamming a radio signal in the UK is illegal and I mention this only for my LE friends who may be able to get the appropriate clearances/warrant to achieve a deauth this way. Of course you would need an antenna faced on the property ready and running Kismet or Airodump to grab the packets as the reauth takes place. I wrote some great Linux Shell scripts to automate the process recently to achieve just this type of situation.

I wont publish where to purchase them, you can always get in touch.

You can also pick up a GSM jammer while you have your credit card out and next time you see in your rear view mirror the lorry driver chatting on the phone whilst passing the local primary school you could have the satisfaction of hitting the button and jamming his call. Oh if only it was legal!!

WPA Cracking

2008-12-18T13:51:00.003Z

In Yorkshire on holiday with the extended family. Touch of man flu!

Its been a while since my last post as life has been flat out. Just a week back I taught the first LE only wireless attack course. I taught it at the Defford SB facility which was perfect, as apart from a bunch of huge radio telescopes there is no wireless interference at all.

What was interesting was the vast difference made by different antenna's. I guess this is obvious but I had the chance to really test the differences between the omni-directional and directional antennas I had available. The out and out winner was the 12dbi directional 'can' antenna which took us to the edge of the facility, at least 100 meters from the Access Point with plenty of power left over. Having returned to the office I thought I would invest in a parabolic mesh antenna slated as 24dbi. I bought 2, one for me and one for an operation I'm working on with a Police force. When they arrived they were HUGE! When put together the dish was at least 70cm square, not terribly useful in a covert setting. When hooked up the coverage was astonishing, I reckon that 1km could be possible with clear line of sight.

As WPA cracking is very reliant on a dictionary attack it is interesting to note that Elcomsoft are releasing a WPA specific cracking tool that uses a dictionary attack associated with GPU acceleration which is very exciting. They have offered me a beta copy and I will let you know how it goes.

The company already has brute force cracking a ability of WPA passphrases with GPU acceleration which the press have been having a field day over, saying WPA is dead. In reality a box with 2 super fast NVIDIA GTX 280 cards in will still take 3 months to break an 8 character password. I think the new dictionary version will be much faster.

We shall see...

Just take what you need!

2008-10-16T20:43:00.002+01:00

Sat in Brussels airport, flight delayed for 2 hours, 10pm :(

I’ve been presenting today at the European Network Forensics and Security Conference in Holland. It is not a big event but there were some very interesting people in attendance including Laura Chappell from Wireshark University and James Lyle from NIST. I had not met either before but look forward to communicating more with them in the future.

I was presenting today on the subject of extracting just the information we perceive we need from a case rather an always imaging an entire drive, or more commonly now, a gaggle, bunch, collection (what is the term for multiple drives) of drives which regularly can exceed a TB. Now I know the purists amongst you will shout foul, the whole drive is best evidence and I do not disagree with you; but when dealing with, for example, a fraud case where the predominant evidence will be found in email, an accounting partition and chat logs, why ‘initially’ image vast amounts of data when we know where to start. It is very straight forward to image out just a .pst file or just take a partition and this can reduce processing and searching times tremendously. This does not mean that you never image the drive, however when we have multiple machines to look at why initially image them all when the pertinent data might be available in key containers.

A number of Police Forces in the UK and I’m led to believe ACPO too are looking at a methodology of pre-imaging triage to try and reduce workloads and backlogs and I am in general agreement with this.

There are a bunch of ways of extracting what you need. On a live machine you can simply write your own script to search a machine and extract just the files you need. For example, open notepad and just enter:-

xcopy "%systemdrive%\documents and settings\*.pst" /h /s

..save the text file as a batch file (myprog.bat) and put it on a USB key or external drive. When you plug the drive in to a machine and run the batch file it will search all folders under documents and settings and copy back any .pst file it finds. Easy as that! You could make a couple of subtle changes and it would find and copy back all the thumbs.db files which you could parse out in Encase, FTK, Vinetto and have a pretty good idea what images were on the machine. Quite handy.

xcopy "%systemdrive%\documents and settings\*bs.db" /h /s

If you want things to feel a bit more ‘forensic’ then use dd on the target system to extract what you need:-

dd if=\outlook.pst of=e:\harvest\outlook.dd conv=noerror

You could use this method with Helix and use either the Windows terminal on a live machine or boot to the swanky new Ubuntu Linux side and do it there. You can then MD5 the file and off you go.

md5sum > md5.txt

The argument is even more compelling with live servers in a corporate environment. Tell a sysadmin that you are going to shut down his email server for 8 hours while you image it and he will go a rather nasty colour. Do a live response and just take the pertinent .edb or whatever, files and everyone is happy and you likely have all you need. The same argument can be made when looking at a RAID array. The ‘Financial Director’ under investigation will rarely, if ever, have access to the RAID controller to hide any data anywhere clever on the array disks. So in that situation, do a live response on his machine and figure out what disk partitions/folders he has access to and just go and get those. Imaging the appropriate partition on a RAID will give you everything you need and saves a shed load of time trying to figure out the striping pattern.

I appreciate this blog entry is overly simplistic and all these decisions should be made on a case by case basis with full comprehension of what is potentially being missed, however the modern investigator should be aware of these techniques and use them where appropriate.

Backtrack 3 on the Asus EEE (that rhymes!)

2008-08-28T16:34:00.001+01:00

I mentioned a few posts ago about the wonders of the tiny Asus EEE. I’ve just had the latest 901 version delivered with 8 hours battery life and an Intel Atom processor. One of the coolest things I’ve been doing is booting the machine to an alternative OS on an SD card. Perhaps one of the most useful is the ability to boot to the Backtrack distro. It means that you have your tiny portable machine totally ready to carry out sysadmin tasks and even wireless cracking using the inbuilt Atheros wireless chipset.

However getting Backtrack 3 to boot on the EEE has been a problem and a number of forums have questions about it. When you download the bootable USB version (http://www.remote-exploit.org/cgi-bin/fileget?version=bt3-usb) there is a helpful text file telling you which files to copy to the USB key or SD card, then simply browse to the ‘Boot’ folder on the card and run the ./bootinst.sh script. To get a command shell up in the EEE Xandros Linux distro just hold down CTRL-SHIFT-T. Then as if by magic you can boot to Backtrack by simply holding down the ESC key at boot time.

However, a number of people have noted that it seems impossible to run the shell script. You simply get an error message. The solution is very simple. If you look at the permissions for the script (ls -la) you will note that the files on the SD card do not have execute permissions. If you try and change the permissions:-

chmod 777 bootinst.sh

..it pretends to work but another look at ls -la and you see that it hasnt.

The problem is to do with the mount permissions for the device as a whole. If you execute the ‘mount’ command you will see that the device is mounted with the noexec flag set and that is what is messing things up! With no other keys or devices plugged in it seems to always mount at /media/D:, so.. simply unmount the device:-

umount /media/D:

then remount with the following:-

mount -o rw /dev/sdc1 /media/D:

Dropping the noexec flag makes the files executable. Now just browse back to the right directory:-

cd /media/D:/boot

then execute the shell

./bootinst.sh

That’s it, now you can reboot to BT3. Have fun.

The way Linux pages data

2008-07-16T14:00:00.002+01:00

I've been spending some more time looking at why the bad sectors on the NIST tests (see last post) were in the middle of a read run. In their conclusions they state that:-

"Up to seven accessible sectors adjacent to a faulty sector may be missed when imaged with dd based tools in the Linux environment directly from the ATA interface."

This doesn't seem to make any sense if we are saying that some sectors are skipped when a bad sector is encountered. Surely it would always be the first sector with later sectors skipped? This explanation seems to go some of the way in finding a solution.

When dd requests a block, the mapping layer figures the position of the data on the disk via its logical block number. The kernel issues the read operation and the generic block layer kicks off the I/O operation to copy the data. Each transfer of data involves not just the block in question but also blocks that are adjacent to the required block. Hence a 4096byte 'page' transfered from the device to the block buffering layer in the kernel (often a page segment in RAM) will contain the bad block and adjacent 'good' blocks.

If you have a 4096byte page with a single 512byte bad block you will have, wait for it, 7 good 512byte blocks in that page. This fits with the observations of NIST that 7 sectors may be missed, obviously something bad is happening to the entire 4096byte page.

They then go on to conclude that:-

"For imaging with dd over the firewire interface, the length of runs of missed sectors associated with a single, isolated faulty sector was a multiple of eight sectors."

This makes perfect sense, as the kernel pages the data in 4096byte blocks including 7 good and 1 bad sectors, any 'loss' of data by the block buffering layer would be in 'whole pages' or 8 sector multiples. Am I making any sense?

Hence, I'm reasoning that when dd hits a bad block, something is happening to the block buffering layer to either overwrite, clear or otherwise remove some or all of the buffered pages. The speed of the differences in moving blocks to and from different media such as ATA rather than firewire may help to explain the different numbers of lost pages. e.g. there is physically more or less data in the buffer when it gets deleted/wiped/overwritten etc.

I now need to look at why the buffer is possibly being affected. Any comments are welcomed!

Link to the NIST research on dd isues

2008-06-24T10:19:00.002+01:00

I've written a couple of simple overviews of the issues surrounding dd and the seeming lost sectors when bad blocks are encountered. I neglected in my previous posts to include a link to the research by NIST at http://dfrws.org/2007/proceedings/p13-lyle.pdf.

Speaking to both Drew Fahey and Barry Grundy the feeling is that there is no reason to overreact, virtually every tool we use has some flaw or another, however further research is needed to be clear about the issue and how to circumvent it.

I'm off to present at the ACPO conference tomorrow and I'm sure the subject will come up, I'll post any interesting comments.

Norway

2008-06-11T19:48:00.002+01:00

I'm teaching this week at the National Police University in Norway and have met some very interesting and talented investigators from various services. What is very interesting is the almost total lack of organised defense experts. It is quite fascinating that most cases with computer evidence rely almost totally upon the prosecution expert with no counter from an alternative position.

As I do both prosecution and defense work I can see the pros and cons from both sides but although I do not doubt the integrity of the officers here I do believe that a sound defense requires experts giving testimony from both sides. Even though with the best will in the world the reports should be the same, we both look at the same data, we all know that things get missed and some issues and elements can be explained in more ways than one.

It is does seem that some officers are now beginning to leave the service and set up on their own so I suppose we will begin to see that change. In the UK, of course, we have many defense experts and although one has to wonder about the competence and even integrity of one or two, at least a defendant can be assured of a second set of eyes on the data. Dont get me started on the need for industry control, I can go on all day. Doesn't mean I know how to solve the problem though!

I guess setting up in Norway could be a good thing for someone?

Linux dd issues part 2

2008-06-11T19:34:00.004+01:00

I spoke in the last few posts about the issues with dd both in Windows and Linux. Having recommended in a previous post that you use dd_rescue with the -d flag added to enable direct disk access I have since found that when running it from the Helix distro it appears to work but instead creates a 0 byte file. I can't get my head around why it would do this.

However, following more research it appears that using GNU-dd in Linux you can enable the iflag=direct argument. This seems to enable O_DIRECT disk access and avoid the seeming buffering issues. Testing this against a drive with no errors it acquired the drive as expected and provided the right hash, so at least it doesn't mess things up.

Interestingly I emailed Barry Grundy about it and he had been following the same line of research and testing. Both of us are away from our labs for a week or so and will not be able to test against a drive with bad sectors until then but I will post again.

If you wish to try it the syntax is simple:-

dd if=/dev/(drive) of = (where you save it) conv=noerror iflag=direct

If you get any interesting results please don't hesitate to contact me.

...and FAU-dd issues

2008-06-03T22:32:00.003+01:00

Having just posted about DCFLDD, my good friend Jim also pointed out that I had ignored the issues with FAU-dd from George Garner. Helix uses this dd version on the Windows side, specifically because it supports the \\.\PhysicalMemory device to grab RAM. It has been noted that even if the block size is set to 512b FAU-dd still copies data at 4096b to increase speed. however, if it encounters a bad block it will skip 4096b.

The latest version from George steps back from 4096b to 512b when a bad block is found to minimize lost data but unfortunately support for \\.\PhysicalMemory was removed in that version. This is only an issue if bad blocks are found. Removing the noerror switch will stop dd if errors are found and enable you to use a different tool if you are concerned about this. (do not remove the noerror switch when imaging RAM, it will stop almost immediately)

Also, to get around this, FTK imager is installed on the Windows side and there are no reported problems of this type with that tool. However, running from a GUI will have a greater footprint on a live system.

DCFLDD problems

2008-06-03T16:57:00.004+01:00

A number of concerns have been raised recently about certain linux dd implementations such as DCFLDD. You can read about it at http://www.forensicfocus.com/index.php?name=Forums&file=viewtopic&t=2557 and http://tech.groups.yahoo.com/group/ForensicAnalysis/message/82

In simple terms the problems revolve around how dd treats a bad sector. With the noerror flag set one would hope that dd would jump the bad sector, zero it and move on. However it would seem that a number of sectors are being missed when a bad block is found. Research by Barry Grundy and others indicates that this is due to the way the Linux kernel buffers data coming from the device being imaged. The buffering is a good thing as it speeds things up but it also would seem to enable the skipping of good sectors when a bad one is encountered.

This affects one of my favourite tools, Helix. Helix uses the DCFLDD tool as a basis for the Adepto GUI on the Linux side. In the meantime if you are using Helix you can make use of dd_rescue, making sure that the -d flag is set which enables direct disk access to the device. If you were planning to image the disk sda to an attached drive sdb1 this would look something like:-

dd_rescue -d -v /dev/sda /media/sdb1/image.dd

The release of Helix Pro later this year will deal with issue.

SMTP woes

2008-05-31T21:27:00.002+01:00

I've recently enjoyed a holiday in France and frighteningly one of the first questions I asked my Brother who booked the house was about Internet availability. He had already asked and Wifi was available in the house. It meant my hands could stop shaking with the stress of possibly being disconnected for 2 weeks. Well in reality my Vodafone dongle would have taken a hammering.

We rocked up to the house (beautiful place by the way) and 20 minutes after unpacking the cars there were 2 MacBook Pros glowing silently on the dining room table. In fact we had 3 notebooks between us as I had also taken my Asus EEE as mentioned in the previous post. Sad eh, but even my wife doesn't moan anymore as long as emails are answered, blogs are written etc at appropriate times.

In fact the laptops came in useful on a number of occasions, looking up the weather, finding a local Kart track, finding a good restaurant and route finding to a Chateaux. Even the parents and in-laws were on board.

Later that day a number of emails arrived but as I've found with a number of ISP's my normal SMTP details were blocked. There are a bunch of ways around this but for your information I used http://whatismyip.com to get the IP address assigned to the router, next I did a look up on SamSpade to find out who owned the IP. This turned out to be France Telecom i.e. Orange, a quick Google search found the details smtp.orange.fr which then worked perfectly with no authentication.

If you travel alot there is a paid option of www.smtp.com, for about $10 a month for 50 emails a day you can send emails through any ISP without the hassle of changing details.

You can of course just switch to webmail but I like my Mac Mail.

As an aside I cracked the WEP code on the house's router in 4 minutes 37 seconds - AAAAAAAAAAAAAFFFFFFFFFFFFF. I love my EEE!

EEE'up its good

2008-05-27T15:04:00.003+01:00

A number of us have been working on the new Asus EEE PC 900. If you haven't heard of it, its a small form PC which is still very useable. The new 900 has a 20 gig solid state HD and larger screen than its predecessor. (I've got the black version which I think looks nicer than the 'ipod'esque white one).

The rather cool element to the EEE is the in-built Atheros WIFI chipset which supports monitor mode and packet injection. I'm not going to write a detailed explanation about why this is a good thing but any user of Aircrack-ng, Kismet or other such tools will be delighted.

The default OS is a Xandros Linux environment which is quite cool for day to day browsing use, however you are able to boot from the internal SD slot. With a little fiddling you can install Backtrack on an SD card, make it bootable (check the readme on the Backtrack download) and just by holding down the ESC key at boot time, fire up a full Backtrack environment. I managed to get up and working in about 10 minutes and even had a USB Railink Wifi adapter up and working too. Its tiny size makes it perfect for Wifi activities when out and about and at around £300 quid it would be rude not to!

Kicking off!

2008-05-27T14:53:00.002+01:00

There are lots of computer forensic blogs out on the interweb some superb and others rather less useful. This aspires to be in the latter category. However as I work with, and have the privilege to train some excellent computer forensic professionals both here and abroad, I often hear about some great pieces of research, new tools and other movements within the industry. If appropriate I will try and post them here.

If you tell me about an idea I promise to check with you before I post here and will never name law enforcement persons unless express permission is gained. As you can tell, this is already an exceptionally boring blog.

If you want to contact me (only about computer forensic topics please) please don't hesitate to do so, either via phone, or from the form you can find on the web addresses in the right column.

That'll do for starters