Extracting recent contacts from OSX Mail

(The original blog post can be found here - http://www.csitech.co.uk/extracting-recent-contacts-from-osx-mail/)

Having spent the best part of the last decade working on Live Forensic techniques I've begun to turn my attention to OSX.  I'm an unashamed MacHead but have not spent much time thinking about ways to extract data from a live machine.

Knowing who a suspect speaks to or emails can be very useful in an investigation and so I've started looking at the email system in OSX.  The inbuilt email app, Mail is very widely used and connects to the OSX Address Book for the management of contact data.  However, tucked away in a SQL Lite table is a large list of 'Recent Contacts', which contains the name and email address of recently contacted people who may or may not be in your standard contacts.

You can see this list by opening OSX Mail and browsing to Window - Previous Recipients.  This opens a box with all the recent contacts, but apart from being able to add the contact to your main contacts, there is no way to export them.

I've written a small shell script to extract the name and email from the SQL table and pop them in a csv file for you.

The code is very simple, just 2 lines:-

echo 'First Name,Surname,Email Address' > ~/Desktop/recentcontacts.csv

This simply writes the column heads to a CSV file on your Desktop

sqlite3 -csv ~/Library/Application\ Support/AddressBook/MailRecents-v4.abcdmr 'select ZFIRSTNAME, ZLASTNAME, ZEMAIL from ZABCDMAILRECENT;' >> ~/Desktop/recentcontacts.csv

This opens the MailRecents SQL file and pulls out the first name, last name and email address, writing them to the CSV file on your Desktop.

Easy!

For ease just drop the file somewhere, 'cd' to it and run - ./recentexport.sh

If it doesn't run you might have a permissions issue so just type - chmod +x recentexport.sh

You can download the tool here.

Hope its useful to you.

iPhone Video Metadata - Tool released

Following the research I posted about the available metadata in iPhone video files, my good friend Robin Wood from www.digininja.com has written a tool to extract the data for you.

You can find the research here and the tool here.

Nick

iPhone Video Metadata

(This is also available on the CSITech website at http://www.csitech.co.uk/iphone-video-metadata/)http://www.csitech.co.uk/iphone-video-metadata/

First question, if you start a sentence with the word iPhone should you captialise the ‘I’, answers on a postcard please.

Second question came from a law firm that I often assist with digital forensics cases.  When an iPhone is used to take a video and then distributed does it contain any device ID information that can be used to trace it back to the original phone?

The answer, somewhat surprisingly knowing Apple, appears to be no, I cannot find any reference to the serial number, IMEI or ICCID numbers within the file although it is possible that the data is there but obfuscated in some way.

Whether there or not, looking at iPhone movie data is very interesting.  We are all used to the vast amount of metadata embedded within a photo but movies are a bit more of a dark area with not much written about it.  The movies are based around the QuickTime file type that is well documented by Apple which can be found here - http://developer.apple.com/library/mac/documentation/quicktime/qtff/qtff.pdf

The filetype is awash with metadata, some which are used by default in the iPhone and many that are not.  Although there does not appear to be anything to specifically identify the iPhone which shot the video there are some useful bits of data which could help.  I have focused on a video shot by an iPhone 5 and then emailed out of the device. 

The QuickTime structure is based around Atoms and Keys.  Atoms are small 4 character tags such as ‘prfl’ for profile, ‘tkhd’ for the track header and many, many more.  There are also keys that are of specific interest to us as they contain the primary metadata that we may want.  The keys are in the ‘mdta’ atom and take the form of ‘com.apple.quicktime.author’, for example.

At offset 0x04 you come across the ‘ftyp’ atom which identifies the type of video to follow.  The iPhone uses QuickTime and so the tag which follows is ‘qt’.

Next is the ‘mdat’ atom which I guess stands for movie data and contains the data related to the movie itself.

Next is the ‘moov’ atom which partly indicates that the movie came from a Mac platform, ie the iPhone.  The ‘moov’ atom has a number of sub-atoms which brings us to the area we are interested in.

Once we pass all the obvious movie data we pick up a ‘keys’ atom which is then followed by metadata identified by the atom ‘mtda’.  The entire section can be seen in the image below.

There are several interesting tags here.

©mak«Apple - This identifies that the movie came from an Apple manufactured device.  Although this might sound obvious we might have a series of videos from a suspects computer that we think he may have taken.  However, if he is an Android and PC user then this would reduce the likelihood that he created them.

©swr«6.1.4 - This is rather useful as it tells us the IOS software version that was installed at the time that the video was taken.  Again, a scenario could be that a suspect accuses his co-defendant of shooting a video but we not that the co-defendants iPhone is running an earlier IOS version.

©day«2013-05-27T21:38:21+0100 – This provides us with the time and date that the video was shot.  Helpfully this date does NOT change when the file is moved, emailed or uploaded.  This provides a solid line in the sand as to when the video was made.  The time is also adjusted from UTC so we see the real world time it was created.

©xyz«+52.5461-002.6371+115.546 – This tag ‘@xyz’ provides GPS location data provided by the GPS chip in the phone.  Although not delimited we can divide it up to provide:-

x - +52.5461

y - -002.6371

z - +115.546 – This appears to be the direction taken from the onboard compass.

This data depends on location data being turned on for Photos in the Privacy tab in Settings.

©mod«iPhone 5 - This is great, it doesn’t just tag the device as an iPhone but as an iPhone 5.  Again this may help us to identify the phone in a case that shot a video.  So we know the video was taken by an Apple iPhone 5 with firmware 6.1.4 on the 27/5/13 at 21:38:21 at a specific location.  That’s not bad information.

All the information is then repeated using different tags as follows:-

mdtacom.apple.quicktime.make

mdtacom.apple.quicktime.creationdate

mdtacom.apple.quicktime.location.ISO6709

mdtacom.apple.quicktime.software

mdtacom.apple.quicktime.model

So can we identify a specific device that shot a video?  Not definitively no, however we may have a case where a number of phones are seized, perhaps a couple of Androids, an iPhone 3 and an iPhone 5.  They may all have the same video on their phones showing illegal activity and be accusing one another of shooting it.  In this case we may have sufficient metadata to pinpoint the culprit.

When I first started looking at this I assumed that it was a purely academic exercise as our normal forensic tools probably report this data but it seems not.  A quick look in FTK with my test video only showed the Operating System dating, created, modified etc and not the embedded video created date.  There was also no extraction of ANY of the metadata we have discussed, no model, firmware, GPS data, anything!  Obviously you can manually work through the Hex to find the tags but it could easily be missed if we don’t know it’s there.

Hope that’s helpful to you? 

Maltego Machines and other stuff

Once again it has been several lifetimes of certain moths since I wrote a blog post.  I have been trying to write the text for my new web site whilst also writing a book.  That's right loyal follower, I am writing a book!  The working title is Weaponizing Open Source Intelligence.  Obviously for those of you in the UK it will be Weaponising!  It should be pretty interesting not only covering advanced Open Source Techniques but how to understand how the data can be 'weaponised' into an attack against you or your organisation.  Should be good!

Anyway, 2 weeks back I taught the first Advanced Open Source Course to international acclaim and applause, well, all the students thought it was epic and enjoyed it.  The highlight seemed to be the real-world exercises where you do everything from hunting down bad guys to planning an attack on a company, loads of fun.

A good chunk of the course is focused on the tools from Maltego, CaseFile and primarily Radium, which, frankly, rocks.  If you haven't seen the tool before take a look at Paterva's YouTube channel at http://www.youtube.com/user/PatervaMaltego.  It is essentially a graphing tool to assist with 'automated' Open Source Intel gathering.

One of the interesting things about Radium is the ability to write your own Transforms (searches) but also to code up your own Machines to essentially daisy-chain commands together so that they run automatically.  

During the course we had a segment given online by Social Engineering Guru, Chris Hadnagy where we discussed the identification of key people within an organisation to create targets for phishing targets and the like.  It can also be useful to identify people who may know eachother for the same purpose.  Obviously we are not teaching this to be able to carry out an actual attack but rather identify vectors can could be used by an attacker against us.

I thought it would be interesting to create a Radium Machine that would accept the input of a Domain, extract 50 or so documents and then rip out the meta data in the documents hopefully giving us real names email addresses and like.  Then we can remove any data that only appears once, working on the principle that we would like to ID people who had authored many documents.  I took a good go at writing it and thanks to Andrew at Paterva he tidied it up and made sure it worked properly.

If you have a version of Radium simply click the Machines tab, Manage Machines, New Machine.  You can type any old rubbish into the dialogue as it will be overwritten by this code anyway.  The code looks like this, simply cut and paste into the code window and press the 'tick' button to compile:-

--------------------------------------

machine(
    "MetadataMachine",
    displayName:"Metadata Machine",
    author:"Nick Furneaux (thanks to Andrew)",
    description: "Finds documents and their metadata for a domain and then deletes any documents where the meta data is not found in more than one document"
    )
{


    start {
           
       
        /* Find all documents and then their Metadata */
       
       
        // Get Documents
        status("Searching for Documents")
        log("Finding Documents....",showEntities:false)
        run("paterva.v2.DomainToDocument_SE",slider:100)
       
        // Get Metadata from Documents
        status("Extracting metadata")
        log("Extracting metadata",showEntities:false)
        run("paterva.v2.DocumentToPersonEmail_Meta")
       

     
        /* Remove all entities that have less than 2 links incoming to the entity*/
       


        //now we select any people,phrases and email addresses
        type("maltego.Person", scope:"global")
        incoming(lessThan:2)
        delete()

        type("maltego.Phrase", scope:"global")
        incoming(lessThan:2)
        delete()

        type("maltego.EmailAddress", scope:"global")
        incoming(lessThan:2)
        delete()
       
       
       
        /* Remove any remaining documents that no longer have children */
       
       
        type("maltego.Document", scope:"global")
        outgoing(0)
        delete()
       
        /* Ask if you would like more work to be done on any extracted email addresses */
       
        type("maltego.EmailAddress", scope:"global")
        userFilter(title:"Choose Email Addresses",heading:"Email",description:"Please select the email addresses you want to do more research on.",proceedButtonText:"Next>")
        run("paterva.v2.EmailAddressToPerson_SamePGP")
       
       
       

    }
}

-------------------------------

The first command that runs, looks at the Domain you have supplied and goes looking for Office or PDF documents posted to that Domain.

     run("paterva.v2.DomainToDocument_SE",slider:100)

Next these documents have their metadata extracted.

     run("paterva.v2.DocumentToPersonEmail_Meta")

Then we remove any metadata that has less than 2 links to it.

       //now we select any people,phrases and email addresses
        type("maltego.Person", scope:"global")
        incoming(lessThan:2)
        delete()

        type("maltego.Phrase", scope:"global")
        incoming(lessThan:2)
        delete()

        type("maltego.EmailAddress", scope:"global")
        incoming(lessThan:2)
        delete()

Lastly, we display any email addresses and ask if you want more work done.  At the moment it just looks at a PGP server and tries to extract the registered name for that email address which could be useful.  We could do a web search for sites containing that address too.

        userFilter(title:"Choose Email Addresses",heading:"Email",description:"Please select the email addresses you want to do more research on.",proceedButtonText:"Next>")
        run("paterva.v2.EmailAddressToPerson_SamePGP")

As code goes, this is pretty simple and can help to automate tasks that you run regularly.  Interestingly the code also enables you to set timers to run the script every minute, hour or whenever.  This could be very useful for monitoring a specific Domain for new activity etc.

Thats all for now.  If you want to learn more about the Advanced Open Source Intelligence Course you can download a syllabus here - www.csitech.co.uk/Advanced_OSI_Syllabus.pdf.

 

 

 

Password extraction fun

Extraction of passwords whether remotely or by gaining physical access to a computer is always an area of interest for my clients.  If you can acquire the Windows password this can be very useful, users often consider their OS to need a very strong password, not realising that they are very easy to crack.  The average number of passwords used by a person tends to not exceed 3, or derivatives of 3.  If you get the Windows password it tends to be the 'strong' one that they use and so applying it to their Paypal, Gmail etc you might be successful.

Generally the way to grab the password is to dump the LM/NT hashes either by grabbing the SAM or from a RAM dump and then use Rainbow tables (or a dictionary or brute force attack) to decrypt the plain text.  This is not terribly hard but requires some knowledge and there is always the possibility of the crack not coming through for you.

Somehow I had missed the release of a tool called Mimikatz written by a chap with an extraordinary ability to undermine security holes within Microsoft (and has a penchant for writing everything in French, tres bien).  If you would like to know how his technique works then please take the time our to read his cracking Powerpoint click here (Thankfully not in French!). Thanks to my friend Jon Evans who mentioned it to me last week.

Mimikatz can achieve a number of things but the most useful to me is its claim to extract plain text user passwords.  Guess what - it works!

Here's what to do.

Download Mimikatz

Run the 32bit or 64bit version as administrator (please dont make me explain how you would know which!!) and you are presented with a console environment.

Next get into debug mode with the command:-

privilege::debug

Next simply dump the passwords by running:-

sekurlsa::logonPasswords full

Job done!

Username - nickfx
Password - 123

Easy eh!

This is an extremely useful addition to any first responder toolkit and I highly recommend having a go for 10 minutes.

Volatility - cmdscan buggy?

I tweeted last week that I was impressed with a new command in Volatility called cmdscan.  The command is designed to extract command shell history.  I had run it on a variety of new and old RAM dumps and appeared to get slightly random results, often interspersed with obviously correct history.

In my tweet I made the comment that the command was good but a bit buggy. 

An example of my issues are in the image below:-

You can see that the upper part of the results seem to display erroneous results whereas the lower portion is very obviously a series of recovered commands. 

To my pleasant surprise Michael Ligh himself dropped me a line asking for more details which I duly provided only to discover that I should have not been such an ass but have checked the code before making the comment,  Turns out it is doing exactly what it should.  I thank Mike for his gracious response and explanation which I re-print here:-

MHL

...so in short, cmdscan is to consoles as psscan is to pslist. In other words, the consoles plugin (not sure if you tried that one) will find active/running console sessions (like pslist will only find active processes) and not only print command history but full input/output buffers. The cmdscan plugin, on the other hand, will scan through memory using pattern matching and try to brute force with sanity checks etc - the advantage being that it can not only find histories from active/running processes but also closed consoles that have been partially deallocated or overwritten (similar to how psscan carves and finds terminated processes).

If you take a look at the command history structure:

http://code.google.com/p/volatility/source/browse/trunk/volatility/plugins/malware/cmdhistory.py#44

You'll see there's a CommandBucket member which is an array of pointers (to command structures). The CommandCount member tells you how many pointers in the CommandBucket are valid. However, if the command history structs belong to closed/terminated processes, then we cannot rely on CommandCount. It could be 0 although there are still valid pointers in the CommandBucket array. Or vice versa - it could be 40 although there are only 10 valid pointers in the array - not even continuous, it could be slots 0, 4, 5, 10, 11, 12, 18, etc.

So cmdscan ignores the CommandCount member and treats CommandBucket as an array of 50 pointers, because 50 is the max history on most systems. If a pointer points to a valid location (i.e. somewhere allocated and not paged) and looks like it might be at lest some unicode characters, then its printed to the terminal.

If you look at your "Screen Shot 2012-09-21 at 15.27.59.png" image, it says CommandCount is 15. You see slots 0-14 are are valid but it goes on to print slot 18, 25, 32, 39, and 46 anyway just in case CommandCount isn't accurate. The consoles command would trust CommandCount and only print slots 0-14.

After looking at the screen shots, I'd say the plugin is working as expected. So if you do get a chance to look over the code, its pretty well commented and you should be able to figure out why it seemed buggy.

Thanks again to MHL and the Volatility team for such a useful toolset.

Advanced Open Source Intelligence Gathering

 The Internet contains a vast amount of information about people that may be of interest to us.  Police and other Agencies may want to know more about a suspect, a company may want to research the background of a senior candidate or understand the ‘exposure’ of their company or key employees.  Much can be gathered if you know how to exploit online resources.

 
This course focuses on the investigators ability to gather information on people, groups or companies from the Internet in a truly advanced manner.  Rather than just using ‘advanced’ Google searches and other web sites we will be leveraging the tools available to look ‘under the surface’ of the internet, accessing data gleaned by understanding database API’s used by the likes of Twitter, Facebook and others, ‘dark net’ data collection methods and other areas rarely taught.  

The 4 day course is completely hands-on and will teach a range of skills from staying anonymous, bouncing data around the world, setting up false online identities, extracting data using API's, using Patervas awesome Maltego and graphing and visualizing data both historical and in real-time.

We have already begun seeding the Internet with the false identities of subjects that we will be investigating on the course.  The final exam will pit your new skills against the online world as you work to discover all you can about a person, their friends and what they are planning to do!

Other Open Source courses are available, but not like this!

The course will include a 6 month license for Maltego Case File, 6 months VPN access, an encrypted 
hard drive, a large number of software tools and course manual.

The 4 day course is £1800 + VAT 

Nick Furneaux (me!) teaches Law Enforcement agencies all over the world and this is the first time that corporate students have been accepted.

To inquire further please contact me here

Syllabus

Day 1

Understanding the law – what can you do?

Setting up your tool kit

            Encryption of data

            To cache or not to cache

            Benefits of using Virtual Machines

            Adding magic to Firefox

Bouncing anonymously round the world – Proxies and VPN’s

Setting up your own false identities

Day 2

Maltego Case File usage

Aggressive searching – only search the part of the web you need to

            Lots of useful sites to bookmark and try

Searching through maps

            Using social media to ‘see’ an area

What can a web site tell us?

            Who owns it and where are they?

            Blowing a web site apart – mapping a web site in real time

            What did it used to say? - Finding deleted data on the Internet

            Finding hidden links

            Finding documents

            The wonder of Metadata!

Maltego V3!

Google Hacking 101

Understanding email – identification and tracking

Day 3

Finding forums, blogs, websites, IRC entries

            Working with IRC clients

Are you or you organization leaking?

            Using your skills to understand your own vulnerabilities

            Checking if hackers have released your/corporate information

Exploiting Social Networking

Mapping Social Networking accounts and followers

Following the network – don’t forget the family!

Extracting data from Twitter via API

Extracting data from Facebook via API

Facebook ‘naughtyness’

Graphing Twitter data LIVE

Day 4

Geo location possibilities (Where are they, or are they where they say they are?)

EXIF data extraction

Plane and Ship mapping

Enumerating Geo-Coordinates using API

Finding people using public records

Being a bit more aggressive to get IP’s

101 Social Networking – why not just call and ask what you want to know!

Final exam – Full online search and enumeration of a named subject.  Course grade based on details located. (Open book)

Course certificated and graded.

TOTAL COST - £1850 + VAT