DeepMac Reboot progress report

I’ve updated the perl code used to archive the IEEE registry files for OUI’s (now relabeled by IEEE but that’s soley for branding purposes so I’m not going to bother to switch my own terminology). The changes were mostly to deal with the new size offered (OUI 28-bit) and some minor formatting changes. Only other change was to move to a recursive directory storage for storing the files to keep things neater.

The real work has ben on all-new code written in Python to support a new approach for storing DeepMac metadata. The classes I’m working on will eventually allow for adding, deleting and changing data in the repository in a journaling style, where all changes are recorded. So even if something is “deleted” there’s still a record of it having existed. This will be quite handy in the future.

Once the code is actually working with an initial filesystem prototype there will eventually be a Web-based and database-based connection supported. My plan is to have ultimately have an API that will make it programatically simple to be able to add new data to the repository, either manually or in an automated way.

 

The final step will be to have the current web-based search system moved to using the new repository on the backend (it’s still using the old MySQL database I originally created in the early alphas). The search engine will let you see the “current” snapshot of all the metadata but also allow a view of historical data.

Stay tuned!

DeepMac database update

The DeepMac database has been updated with the most recent date information as of December 2nd, 2009. I also updated the MySQL dump of the database.

I haven’t had time to add further fingerprints to DeepMac, nor expand on tools for searching and updating. But please do not hesitate to submit ideas, suggestions or comments!

(Also, I like cookies)

 

DeepMac ALPHA

Almost a year after I first wrote about my idea for DeepMac, I can finally show something actually functional for the world to point and stare at!

DeepMac ALPHA is available at http://deepmac.jedimercer.com/ and includes the full MySQL dump of the database contents.I’m licensing the database under the Open Database License.

At this point, there’s only a few hundred OUI’s tied to a device of some sort. This is because I’ve had to focus on developing the skeletal framework to actually store the information, and have not yet put together tools for automating any part of the process of submitting data. Still, there’s a good breakdown of virtual computers, network-attached cameras, the major Smartphone models and a few videogame systems.

For DeepMac to become truely useful though, it needs a lot more raw data, and not just the easy pickings. That means there needs to be better ways to get data into DeepMac. A primitive search interface is just a baby-step. Ideally, we need to have a standard format for submitting new DeepMac data for analysis and inclusion. Mostly this is about sitting down and figuring out what would work best.

The other thing DeepMac needs to be useful is of course a real-world application that everyone can benefit from. I’d love to see DeepMac data integrated with great security tools like Wireshark, nMap, Kismet and others. There’s also a ton of commercial security software that could benefit from DeepMac such as SIMs, asset management tools and anything else that does network discovery. Alas, I don’t see that happening anytime soon. A side-project of mine is developing a tool for tracking IP<->MAC pairings for network surveillance. I’ll be integrating DeepMac data into that tool, and hopefully releasing it as an Open Source project sometime next year.

In the meantime, I’m still very much interested in feedback about the DeepMac project! What I’ve gotten so far has been small but very useful and greatly appreciated. And if you have any intelligence to offer the DeepMac project (as in OUI Device mapping), please e-mail me!

DeepMac Progress

It’s been almost exactly one month since my last entry on DeepMac’s goals and I’m happy to say significant progress has been made!

The biggest achievement is getting the project finally organized, with clear goals, usable data and a skeletal framework. But that’s kind of broad, so let’s breakdown what has been achieved:

  1. Historical creation dates for OUIs have been documented going back to 1998, thanks mostly to archive.org and the magic of Google. Thus the creation of the DeepMac knowledgebase!
  2. An automatic archiving of the IEEE OUI listing in text format has been established, running on a daily basis.
  3. Simple perl scripts have been written to convert the IEEE OUI file (oui.txt) into a tab-delimited format, one OUI per line, and to add OUI creation dates from the DeepMac knowledgebase.
  4. An actual MySQL database has been created and another perl script used to load some of the knowledgebase data into the MySQL database.
  5. A horribly primitive PHP interface has been written to allow very basic searching of the DeepMac database.

There is still an enormous amount to accomplish with this project. I’ve been thinking about where the focus should be and while I really want to evangelize the project and get input from the big boys in networking, it seems to me that it will never be managable until the project is fully organized and has the tools it needs. So the push for now is going to be on continuing to refine the MySQL database for DeepMac in terms of structure, and lots of work on a web-based front-end.

Ideally, it should be possible to not only fully search DeepMac on-line but also submit information, and for administrators to update the contents including approving submissions for inclusion. That’s a tall order, so the first sub-goal here will be to whip the search inteface into shape and allow full searching on multiple fields with varying levels of output detail.

 

DeepMac Goals

DeepMac has one primary goal, to provide as much intelligence about MAC addresses as possible. While IEEE provides a simple directory of OUI (aka MAC prefix), we want more. Here I’ll outline the milestones for this goal, with the realization that it is unlikely that all OUI assignments will reach the level of detail desired.


Creation Dates

I’ve verified with IEEE officials that the assignment date for each OUI is on record. However, that information is not freely available. IEEE officials did inform me that an individual organization may submit a request to find the creation date for a particular OUI assigned to them. Oh, and why do we want creation dates? It can be a very rough indication of the age of a device. For example, a CISCO device with a MAC prefix assigned in the 1990’s is more than likely an elderly model.

Device Classification

The real meat of the DeepMac project is to try and classify as many OUI’s as possible by the device the MACs are assigned to. For a very large number of MAC prefixes, these will be network interface cards, but in the past 10 years we’ve seen an explosion of embedded networking in consumer electronics, vehicles, and industrial devices. Device classification will be the most research-intensive aspect of the project, requiring a lot of searching, requests for assistance, and verification via input from device owners.

Vendor Participation

It is not likely that DeepMac will succeed in its goals without participation from major vendors such as Cisco, IBM, Apple, and Nokia.  Significant blocks of the OUI space are assigned to a relatively small number of companies, and breaking those blocks down for device classification will require “insider” knowledge about how the OUI space was parceled out by the vendor. I intend to reach-out to the security community at large, but also to folks within these large stakeholders.

Project Simplification

Most importantly, the DeepMac project can’t succeed if it is not kept simple. There needs to be relatively little effort in order to contribute information. The OUI blocks need to be consolidated for research purposes, so the largest blocks owned by a single company can be focused on. Tools for automatically converting and extracting data related to OUI and MAC prefixes should be developed. And finally, the end-result of all this research and data compilation needs to be a simple, clean data file that can easily be parsed or re-compiled for use by various tools.

 


So here’s the pitch: I need help! What I’m currently looking for is participation, in the form of people with first-hand knowledge about a particular MAC prefix or company with OUI assignments. I’m also looking for contributions of device classifications. The Cisco OUI space is going to be a big priority, both due to size and the utility of the additional intelligence. So if you work for a major company that owns OUI space, please drop me a line! General suggestions and comments are also welcome, of course.

 

You may e-mail me at jedi@jedimercer.com

MACOUIIAB!

Quick, what does MAC stand for?

Even after decades working with computers, I still forget. No one really remembers “Media Access Controller”, because we call them NICs. MAC is just short for “MAC Address”. Frankly, more often than not I think of MAC as “Machine Address Code”. Makes more frelling sense to me.

But for those who have no clue what I’m talking about, MAC does indeed stand for “Media Access Controller”, it relates to Network Interface Cards and we don’t really care about Media Access Controllers because no one calls them that, we call them NICs or Network Cards. Got that?

Here’s what everyone in IT means when they refer to a MAC: 00:11:AA:44:F1:DD

Every network controller needs a MAC address, regardless of what kind of networking it does. MAC addresses are need for layer 2 networking. If you don’t know what that means, you can review the OSI model or just ignore it because it’s not important. Just know that MAC addresses (or MACs) are just as critical to computer networks as IP addresses. And just like IP addresses, MACs are assigned by a central authority. The IEEE.

<cue ominous crack of thunder>
(more…)