Dataset Cropper for SLAM

Revised September 1, 2014, based on a code begun July 16, 2009.  
This code is copyright © 2014 by Vincent S. Cronin.

startTime=AbsoluteTime[];

Introduction

The purpose of this notebook is to minimize a large DEM dataset so that it can more efficiently be processed in SLAM.

Code

Test file

Change the style of the next line from Text to Input to test the subsequent code without having to use an external data file.  Make sure the code that is normally used to import a data file (in the gray box in the "Input the data file" section below) is disabled during the test -- disable it by converting the Import code line from an Input style to a Text style.

initialDataFile = {{"ncols", 6}, {"nrows", 7}, {"xllcorner", 0}, {"yllcorner", 0}, {"cellsize", 10}, {"NODATA_value", -9999}, {11, 12, 13, 14, 15, 16}, {21, 22, 23, 24, 25, 26}, {31, 32, 33, 34, 35, 36}, {41, 42, 43, 44, 45, 46}, {51, 52, 53, 54, 55, 56}, {61, 62, 63, 64, 65, 66}, {71, 72, 73, 74, 75, 76}};

Choose the percentages of the input DEM to be trimmed off the top, bottom, right and left sides

topPercentage=40;

bottomPercentage=32;

rightPercentage=50;

leftPercentage=35;

Input the data file

This code reads an external data file that is in the form of an ASCII DEM file, with the .dat file-type designator.  On Vince Cronin's office Macintosh, the Import code would look like the following for a data file named "tahoedem.dat" located on the desktop:

initialDataFile = Import["/Users/vince/Desktop/tahoedem.dat"];

On the Dell computer in Cronin' s office, the Import code would look like the following for a data file named "tahoedem.dat" located on the C drive in the "TahoeRuns" directory :

SetDirectory["C:\TahoeRuns"];

initialDataFile = Import["tahoedem.dat"];

Use the appropriate Import code below, and make sure it is in the Input style.

initialDataFile = Import["/Users/vincecronin/desktop/largetahoe.dat"];

origNcols=initialDataFile[[1,2]]

origNrows=initialDataFile[[2,2]]

origXLLCorner=initialDataFile[[3,2]]

origYLLCorner=initialDataFile[[4,2]]

cellsize=initialDataFile[[5,2]]

9.3770677923454`

nodataValue=initialDataFile[[6,2]]

Make the hillshade image

The adjustContrast value forces one point of the DEM to have a particular value, which in turn allows the user to adjust the tonal range of the hillshade map.

adjustContrast=0;

npDipTr=5;

demImgFile = Table[initialDataFile[[i + 6, j]] , {i, origNrows}, {j, origNcols}];

elev3 = Table[If[((i > 1) ∧ (j > 1)), demImgFile[[i, j]], adjustContrast], {i, origNrows, 1, -1}, {j, origNcols}];

npDipTrRad=npDipTr*(π/180);

lightDirection = If[npDipTrRad>(3π/2),(2π)-npDipTrRad,(π/2)-npDipTrRad];

hillshadeMap1 = ReliefPlot[elev3, ColorFunction -> "GrayTones", LightingAngle -> {lightDirection, Pi/12}];

Show[hillshadeMap1]

ClearAll[elev3];

Extract the reduced data set from the original file

initialHeaderData=Table[initialDataFile[[i,j]],{i,6},{j,2}];

fatDEMdata=Drop[initialDataFile,6];

Take a percentage off the top, bottom and sides

haircut=IntegerPart[origNrows*topPercentage*0.01]

haircutDEM=Drop[fatDEMdata,haircut];

Clear[fatDEMdata];

stripBase=-1*IntegerPart[origNrows*bottomPercentage*0.01]

yllcorner=origYLLCorner-(cellsize*stripBase)

stripBaseDEM=Drop[haircutDEM,stripBase];

Clear[haircutDEM];

matrixSize1=Dimensions[stripBaseDEM];

nrows=matrixSize1[[1]]

stripRight=origNcols-Round[0.01*rightPercentage*origNcols]

stripLeft=origNcols-Round[0.01*(100-leftPercentage)*origNcols]

xllcorner=origXLLCorner+(cellsize*stripLeft)

finalDEM=Table[stripBaseDEM[[i,j]],{i,nrows},{j,stripLeft,stripRight}];

Clear[stripBaseDEM];

matrixSize2=Dimensions[finalDEM];

ncols=matrixSize2[[2]]

elev3 = Table[If[((i>1)∧(j>1)),finalDEM[[i, j]],adjustContrast], {i, nrows, 1, -1}, {j, ncols}];

hillshadeMap2 = ReliefPlot[elev3, ColorFunction -> "GrayTones", LightingAngle -> {lightDirection, Pi/12}];

Show[hillshadeMap2]

ClearAll[elev3];

headerData={{initialHeaderData[[1,1]],ncols},{initialHeaderData[[2,1]],nrows},{initialHeaderData[[3,1]],xllcorner},{initialHeaderData[[4,1]],yllcorner},{initialHeaderData[[5,1]],initialHeaderData[[5,2]]},{initialHeaderData[[6,1]],initialHeaderData[[6,2]]}};

mergedData=Join[headerData,finalDEM];

Export the resultant data file

Choose a name for the resultant data file (different from the name of the original data file) and specify the path to the directory where that file is to be saved.  For example, on Vince Cronin' s office Macintosh, the following code will save a resultant file named "thinTahoeDEM.dat" on the desktop :

Export["/Users/vince/Desktop/thinTahoeDEM.dat", mergedData];

on Vince Cronin' s Dell, the Export code would look like the following for a data file named "thinTahoeDEM.dat" located on the C drive in the "TahoeRuns" directory :

SetDirectory["C:\TahoeRuns"];

Export["thinTahoeDEM.dat", mergedData];

Use the appropriate Export code below, and make sure it is in the Input style

Export["/Users/vincecronin/Desktop/croppedDVFZ.dat", mergedData];

How long did this program take to run, in minutes?

minutesForProcessing=(AbsoluteTime[]-startTime)/60