Many forensic artifacts can be looked at from many different angles. A few years ago I proposed a concept of filighting that tried to solve a problem of finding unusual, orphaned and potentially malicious files dropped inside directories that contain files that DO NOT reference these orphaned files at all.

I really hope that forensic analysis tools will evolve to add more features that will help to automate file system analysis based not only on a list of known hashes and/or file extensions, but also paths, partial (relative) paths, file names, actual file types based on their content, and ideas that rely on more complex algorithms: using prebuilt artifacts collections, leveraging various correlations (ideas like filighting), and of course machine learning and AI.

Today I want to explore one more angle of looking at file system artifacts — classes of file content. There are many file formats out there: executables, documents, configuration files, database files, and many other file types. The classification I am focusing on today though is slightly different – the format itself doesn’t interest me too much, but the function of the file does…

My guinea pig will be a license file. The type of a file that is all over the place, but no one reads them. And yes, removing them from the examiner’s view (during file system analysis) may not add a lot of value, but it’s used here only to illustrate the idea. There are many other file classes like this that can be classified as noise to the examiners’ eyes and if we start clustering them together, who knows, maybe we have just saved some personhours there…

I asked myself the following question:

– having a file system in front of me, how do I find all license files on it?

There are at least a few approaches I can think of:

• use hashes of known license files,
• use file names typically used by license files,
• analyze content of all files and look for content that resembles a license file.

All of them have their own challenges:

• the first one needs a lot of prep work to collect good hashes,
• the second one is hard to do w/o some proper analysis of a clean sampleset, and
• the third one is the most reliable, but it’s slow & needs even more preparation because it has to take into account a few more aspects: localization issues (license in various languages), file encoding issues (Unicode variants, ASCII, MBCS), file formats (TXT, RTF, HTM(L), PDF, DOC(X), etc.), and of course — performance (reading many files to analyze their content is expensive, plus not every file referencing GPL, LGPL, GNU is a license file)

I am going to focus here on the second one.

Your typical license file is usually called license, license.txt, eula.txt, and in case of Open Source, we often see files named like gpl.txt, license.gpl.txt, lgpl.txt, etc.

When you start researching this file naming bit a bit more, you will soon realize that there are a lot of variations. A lot of issues listed in 3rd point come to play as well f.ex.:

• file names can be localized,
• file extensions can be .txt, .rtf, .htm(l), .doc(x), .pdf, .xml,
• some of the file names have typos,
• many license file names use various prefixes or suffixes that identify the licensed software, the language or code page identifying the language the license file is written in,
• some file names may refer to compressed file names f.ex. *.tx_ (in installation packages),
• some license files may be stored inside the archives (including password-protected files) or installers,
• some licenses are embedded inside the compiled help files (.hlp, .chm),
• some programs may be hiding the licensing information in files named with various infixes: copying, releasenotes, thirdparty, copyright, and their variants, etc.,
• some may refer to software version in terms of full, trial f.ex. evaluation,
• some files with a license in name often refer to actual software licensing (getting keys, subscription, transferring the licenses, etc.),
• finally, some file names may be available in a 8.3 DOS notation only.

As usual, the more you look, the more complex the problem you see.

For this post I have compiled a large file containing possible license file names. You can download it here.

Will it make anybody’s life easier?

I don’t know.

What matters is that we learned a little bit more how difficult the process of automated file system analysis is. What started as a trivial and frivolous idea ended up being a Don Quixotish attempt to formalize something that is impossible to tackle, even with a data-heavy approach…