CNN on the CHPC

Looking at outputs from last submit to kingspeak trying to get my CNN to work. Checking the memory. I notice memory looks fine, however the shape of X looks wrongfully configured. I’m going to print out the shape of X and such.

[u0949991@kingspeak2:weat_ml]$ cat out/fd_pedestal_cnn.out
Beginning job on kp001 on Thu Feb 28 14:54:19 MST 2019
Job number: 6821264
Running on nodes: kp001
Clean and Load Enviroment
Preparing Local Scratch
Copying Vectorized Data and Labels to Local Scratch
insufficent_data.txt
noisy_data.txt
weather_training_data_v_rnn_extended.npy
weather_training_data_v_rnn.npy
weather_training_data_v_rnn_short.npy
weather_training_labels_v_rnn_extended.npy
weather_training_labels_v_rnn.npy
weather_training_labels_v_rnn_short.npy
Begin FD Pedestal Weather CNN Model Training
Loading Vectorizated Training Data...
Process Memory :	19.8G
MEMORY
------
Total      :   62.9G
Available  :   40.1G
Percent    :    36.2
Used       :   20.9G
Free       :    1.5G
Active     :   39.4G
Inactive   :   20.2G
Buffers    :      0B
Cached     :   40.4G
Shared     :   81.2M
3962
3962
[[[0.]
  [0.]
  [0.]
  ...
  [0.]
  [0.]
  [0.]]

 [[0.]
  [0.]
  [0.]
  ...
  [0.]
  [0.]
  [0.]]

 [[0.]
  [0.]
  [0.]
  ...
  [0.]
  [0.]
  [0.]]

 ...

 [[0.]
  [0.]
  [0.]
  ...
  [0.]
  [0.]
  [0.]]

 [[0.]
  [0.]
  [0.]
  ...
  [0.]
  [0.]
  [0.]]

 [[0.]
  [0.]
  [0.]
  ...
  [0.]
  [0.]
  [0.]]]
0.0
Process Memory :	19.9G
MEMORY
------
Total      :   62.9G
Available  :   40.0G
Percent    :    36.3
Used       :   21.0G
Free       :    1.4G
Active     :   39.5G
Inactive   :   20.2G
Buffers    :      0B
Cached     :   40.4G
Shared     :   81.2M
Finished FD Pedestal Weather CNN Model Training
Copying Model from Local Scratch to General Scratch
0	/scratch/local/u0949991/out/
weather_ml_model_v_cnn.json
weather_ml_weights_v_cnn.h5
Cleaning Up Local Scratch
Job completed on Thu Feb 28 14:56:32 MST 2019

Shapes of Arrays

print X.shape

print len(X), len(y), len(onehot_y)
print X[0].shape, y[0].shape, onehot_y[0].shape
print X.shape, y.shape, onehot_y.shape

(3962, 32, 96)

3962 3962 3962
(32, 96, 1) () (3,)
(3962, 32, 96, 1) (3962,) (3962, 3)