Master Objects Migration Project Plan Home

Scope

This analysis accounts for files noted as "Yes" in the above table that are in the Dark Archive. It does not currently account for those in the J-Drive or External Media.

Detailed data can be found in: fileTypeCount-20151105-rev-dwn-20151106.xlsx

• fileTypeCount-20151105-rev-dwn-20151106.xlsx
• fileExtensions-dwn-20151216.xlsx
  

Collections

For this analysis of materials in the Dark Archive, a report was run on 11/05/2015. The first level sub-folder within a department folder is considered a "collection." While this may not be always true it at least gives us a starting point in attempting to quantify the scope and its impact.

The interesting aspect here is that 38% of the collections (via 3 departments) account for 89% of the files, while 2 departments that have 57% of the collections only account for 7% of the files:

Formats

In seeking to determine what file types are in the Dark Archive a report was run (11/05/2015) to identify the quantities of common file types we may expect to migrate:

• A/V: avi, m4v, mov, ,mp3, mp4, mpeg, mpeg2 and wav
• Database: accdb and mdb
• Document: doc, docx, ,pdf, rtf, txt and wpd
• Image: bmp, jp2, jpeg, jpg, png, tif and tiff
• Presentation: ppt and pptx
• Spreadsheet: xls and xlsx
• Web: htm and html
• xml     
• zip

Of the 1,963,990 files found in the "In Scope" folders in the Dark Archives 1,842,122 (or 94%) are the common file types identified above. A subsequent query (12/16/2015) was run to identifying many of the remaining 6% of files types:

• A/V: MOD, MPG, mpgindex, prev, swf. vprj, wma, wmv
• Compressed: gz, tga
• Database: MDB, sql
• Document: DOC, PDF
• Image: BMP, BridgeCache, BridgeCacheT, BridgeSort, gif, GIF, ico, jpe, psd
• Presentation: thm
• Programs: exe
• Spreadsheet: csv
• Web:css, js, php, url
• XML
• System: bak, dat, db, dll, DS_Store, info, ini, INI, log, LOG, md5, tmp
• NEED TO BE IDENTIFIED:
  
  • There are at least 10 instances of each of these file types: 1, 2, 97, 98, ADD, asp, aub, auf, cct, chm, cof, cop, cos, cot, CR2, CTG, dv, ecf, EMA, eml, FAX, frf, HLP, ICM, IIQ, inc, indd, indt, inf, inx, JUN, ldif, lnk, mbx, mht, MOI, MRK, MTS, mxf, MXF, NEF, NEW, old, orig, out, pct, pl, prproj, RES, sdf, sh, sid, SIF, snm, SUM, swa, toc, u3p, VOB, xmp
  • There are an additional 559 file types that have less than 10 instances each:
    
    • 9 instances - 10 file types
    • 8 instances - 27 file types
    • 7 instances - 3 file types
    • 6 instances - 8 file types
    • 5 instances - 15 file types
    • 4 instances - 98 file types
    • 3 instances - 21 file types
    • 2 instances - 236 file types
    • 1 instance - 141 file types

(NOTE: These quantities are a moving target as files are being disposed of as de-duplication is ongoing and Master Objects are being added as projects are completed)

Format Counts

Of the 1,963,990 files found in the "In Scope" folders in the Dark Archives 1,842,122 (or 94%) are the common file types identified above (as of the 11/05/2015 report).

(NOTE: These quantities are a moving target as files are being disposed of as de-duplication is ongoing and Master Objects are being added as projects are completed)

Format Analysis

By far the TIFF images (1,016,753: tiff = 1,012,982 and tif = 3,771) are the largest group (55%) of format types that will need to migrated.  It can be presumed that these are Master Objects.

The next largest group are JPEGs (314,290: jpg = 299,652; jpeg = 1,307; and jp2 = 13,331). We will need to determine if these are Provisional Masters, or whether they are derivatives (or some other category) that we would not migrate.

The third largest category is XML with 240,714 objects.  173,037 (72%) of these are metadata for the President's Paper in ARV; 45,810 (19%) are in KB; 18,157 (8%) in DI; and the remaining 1% in all the other collections/departments. 

The last significantly large file type is PDF with 216,726 files of which 172,938 (80%) are the President's Paper in ARV. We will need to determine if the remaining 20% are Provisional Masters, or whether they are derivatives (or some other category) that we would not migrate.

There are 20,508 (doc = 4,041; docx = 38; rtf = 37; txt = 16,312; and wpd = 80) documents. We will need to determine whether these are Master Objects, metadata resources, project documentation or non-migration materials.

Of particular concern are 15,149 ZIP files. 15,098 (99.7%) are in DI, which means these are likely from vendors, have been unzipped and can be discarded. The remaining 51 will need to be examined to see if they have been unzipped and whether they duplicate existing materials.

There are 6,413 web pages (htm = 592 and html = 5,821) of which 4,056 (63%) belong to MUS.  The concern with all of these is whether they are derivative exhibit like materials or as is the case with some of the DI stuff it is processing notes supplied by a vendor.

There are 91 spreadsheets (xls = 78 and xlsx = 13); are these metadata, Master Objects or other materials?

Master Objects Migration Project Plan Home