Implement the extendOSM,getImgMat,loadTileXY.
To do:
fix the row,column dismatch of two system(OSM and Matlab).
fix the 0 and 1 starting count.
validate the result.
The version that need to optimize.
Implement data pre-process before PCA.
Add some new file.
Move the params.xymn's initial to initParams.m
Modify the rasterizeOSM stage to use memmapfile
Modify the extendOSM.m to use memmapfile implementation.
The problem is it runs too slow.
Implement the genRawX to extract the feature vectors from the map tile
data, it's so slow, run roughly 4 hours in debug mode.
And the file to store the RawX feature matrix is huge.
With uint8 precision, it's about 40GB for the debug mode data.
Implement the genRawX.m genY.m calMeanStd.m to output the RawX.dat,
RawY.dat, RawXmean.mat and RawXStd.mat
create loadOSMRange.m, modify the initParams.m to change the OSMRange
load approach.
modify the runRec.m and rasterizeOSM.m to fix little bugs.
create some new function file to implement the Neural Network Training.
NNTrain.m the main training file.
getDataByIndex.m, load the X,Y data by the data order.
nnCostFunction.m, this is the function to calculate the cost and
gradient based on the input NN structure and params matrix with input
data case X,Y.
randInitializeWeights.m, this is the function to initialize the
appropriate theta weight matrix based on the input unit number and
output unit number.
sigmoidGradient.m, this is the function to calculate the derivative of
sigmoid function.
modify the getDataByIndex to vectorize version.
And modify the NNTrain to batch operation.
This commit revert the extendOSM.m and rasterizeOSM.m to use only memory
but not memmapfile to handle with its mission.
And have comment the parameters of getImgMat function.
modify the genRawX.m to use only memory and save/load instead of
memmapfile.
modify the genY.m to use only memory and save/load instead of memmapfile.
Implement the y2osmMat.m to transform y to osmMat.
Implement the calAccuracy.m to valuate the metric.
Notice:The current code version cannot run on my own laptop because it
nearly consume 30GB memory that is much larger than my 8GB. So the code
need to be perform on the Server.
add NNTrain_mem.m, predict.m, runRec_mem.m to implement the train and
test stage.
remove useless code in genRawX.m
Replace the constant string file path to the variables in params.
Add the fmincg.m to calculate the Theta Matrix.
fix little issue in runRec_mem.m
remove some useless code and release some useless temp variables.
fix issues in initParams.m
Add some code to save the predict OSM Matrix to file.