Models and layers have special method for that purpose:

    model.count_params()

Also, to get a short summary of each layer dimensions and parameters, you might find useful the following method 

    model.summary()


    import keras.backend as K

    def size(model): # Compute number of params in a model (the actual number of floats)
        return sum([np.prod(K.get_value(w).shape) for w in model.trainable_weights])





Tracing back the `print_summary()` function, Keras developers compute the number of trainable and non_trainable parameters of a given `model` as follows:

    import keras.backend as K
    import numpy as np
  
    trainable_count = int(np.sum([K.count_params(p) for p in set(model.trainable_weights)]))

    non_trainable_count = int(np.sum([K.count_params(p) for p in set(model.non_trainable_weights)]))

Given that `K.count_params()` is defined as `np.prod(int_shape(x))`, this solution is quite similar to the one of Anuj Gupta, except for the use of `set()` and the way the shape of the tensors are retrieved.



 

After creating your network add:   model.summary    
And it will give you a summary of your network and the number of parameters.

How can I write a template literal in ECMAScript&amp;nbsp;6 that will contain backticks(`) in and by itself, (i.e. nested backticks)?

For example:

    var query = `
      UPDATE packet
      SET
      `association` = &quot;3485435&quot;,
      `tagname` = &quot;associated&quot;
     `

----

The reason I need it:

It&#39;s quite obvious in my code example above.

I&#39;m trying to build [node-mysql][1] queries as `Strings` and store them in a variable for passing them to MySQL. The MySQL query syntax requires back ticks for `UPDATE`-style queries.

- The only way I can have them look neat &amp; tidy is by using template literals, otherwise the queries using regular single-line strings look awful because they end up being very long is some cases.

- I also want to avoid terminating lines using `\n` as it&#39;s cumbersome.

  [1]: https://github.com/felixge/node-mysql





Template literals with nested backticks(`) in ES6

How to update mysql data in bulk ?
How to define something like this :

    UPDATE `table` 
    WHERE `column1` = somevalues
    SET  `column2` = othervalues

with somevalues like :

    VALUES
    	(&#39;160009&#39;),
    	(&#39;160010&#39;),
    	(&#39;160011&#39;);
and othervalues :

    VALUES
    	(&#39;val1&#39;),
    	(&#39;val2&#39;),
    	(&#39;val3&#39;);

maybe it&#39;s impossible with mysql ?
a php script ?

Bulk update mysql with where statement

For a Feedforward Network (FFN), it is easy to compute the number of parameters. Given a CNN, LSTM etc is there a quick way to find the number of parameters in a keras model?


How to find Number of parameters of a keras model?

<p>For a Feedforward Network (FFN), it is easy to compute the number of parameters. Given a CNN, LSTM etc is there a quick way to find the number of parameters in a keras model?</p>


I&#39;m studying *TensorFlow* and how to use it, even if I&#39;m not an expert of neural networks and deep learning (just the basics).

Following tutorials, I don&#39;t understand the real and practical differences between the three optimizers for loss. I look at the [API][1] and I understand the principles, but my questions are:

 **1. When is it preferable to use one instead of the others ?**

 **2. Are there important differences to know ?**


  [1]: https://www.tensorflow.org/versions/r0.7/api_docs/python/train.html#optimizers

Gradient Descent vs Adagrad vs Momentum in TensorFlow

Considering the [example code][1].

I would like to know How to apply gradient clipping on this network on the RNN where there is a possibility of exploding gradients.

    tf.clip_by_value(t, clip_value_min, clip_value_max, name=None)

This is an example that could be used but where do I introduce this ?
In the def of RNN 

        lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
        # Split data because rnn cell needs a list of inputs for the RNN inner loop
        _X = tf.split(0, n_steps, _X) # n_steps
    tf.clip_by_value(_X, -1, 1, name=None)
    
But this doesn&#39;t make sense as the tensor _X is the input and not the grad what is to be clipped? 

Do I have to define my own Optimizer for this or is there a simpler option?



  [1]: https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3%20-%20Neural%20Networks/recurrent_network.py

How to apply gradient clipping in TensorFlow?

Using Anaconda Python 2.7 Windows 10.

I am training a language model using the Keras exmaple:

    print(&#39;Build model...&#39;)
    model = Sequential()
    model.add(GRU(512, return_sequences=True, input_shape=(maxlen, len(chars))))
    model.add(Dropout(0.2))
    model.add(GRU(512, return_sequences=False))
    model.add(Dropout(0.2))
    model.add(Dense(len(chars)))
    model.add(Activation(&#39;softmax&#39;))

    model.compile(loss=&#39;categorical_crossentropy&#39;, optimizer=&#39;rmsprop&#39;)

    def sample(a, temperature=1.0):
        # helper function to sample an index from a probability array
        a = np.log(a) / temperature
        a = np.exp(a) / np.sum(np.exp(a))
        return np.argmax(np.random.multinomial(1, a, 1))


    # train the model, output generated text after each iteration
    for iteration in range(1, 3):
        print()
        print(&#39;-&#39; * 50)
        print(&#39;Iteration&#39;, iteration)
        model.fit(X, y, batch_size=128, nb_epoch=1)
        start_index = random.randint(0, len(text) - maxlen - 1)

        for diversity in [0.2, 0.5, 1.0, 1.2]:
            print()
            print(&#39;----- diversity:&#39;, diversity)

            generated = &#39;&#39;
            sentence = text[start_index: start_index + maxlen]
            generated += sentence
            print(&#39;----- Generating with seed: &quot;&#39; + sentence + &#39;&quot;&#39;)
            sys.stdout.write(generated)

            for i in range(400):
                x = np.zeros((1, maxlen, len(chars)))
                for t, char in enumerate(sentence):
                    x[0, t, char_indices[char]] = 1.

                preds = model.predict(x, verbose=0)[0]
                next_index = sample(preds, diversity)
                next_char = indices_char[next_index]

                generated += next_char
                sentence = sentence[1:] + next_char

                sys.stdout.write(next_char)
                sys.stdout.flush()
            print()

According to Keras documentation, the `model.fit` method returns a History callback, which has a history attribute containing the lists of successive losses and other metrics.

    hist = model.fit(X, y, validation_split=0.2)
    print(hist.history)

After training my model, if I run `print(model.history)` I get the error:

     AttributeError: &#39;Sequential&#39; object has no attribute &#39;history&#39;

How do I return my model history after training my model with the above code?


**UPDATE**

The issue was that:

The following had to first be defined:

    from keras.callbacks import History 
    history = History()

The callbacks option had to be called
    
    model.fit(X_train, Y_train, nb_epoch=5, batch_size=16, callbacks=[history])

But now if I print

    print(history.History)

it returns

    {}

even though I ran an iteration. 

How to return history of validation loss in Keras

I found in many available neural network code implemented using TensorFlow that regularization terms are often implemented by manually adding an additional term to loss value.

My questions are:

1. Is there a more elegant or recommended way of regularization than doing it manually?

2. I also find that `get_variable ` has an argument `regularizer`. How should it be used? According to my observation, if we pass a regularizer to it (such as `tf.contrib.layers.l2_regularizer`, a tensor representing regularized term will be computed and added to a graph collection named `tf.GraphKeys.REGULARIZATOIN_LOSSES`. Will that collection be automatically used by TensorFlow (e.g. used by optimizers when training)? Or is it expected that I should use that collection by myself?

How to add regularizations in TensorFlow?

What is the difference between &#39;SAME&#39; and &#39;VALID&#39; padding in `tf.nn.max_pool` of `tensorflow`?

In my opinion, &#39;VALID&#39; means there will be no zero padding outside the edges when we do max pool. 

According to [A guide to convolution arithmetic for deep learning][1], it says that there will be no padding in pool operator, i.e. just use &#39;VALID&#39; of `tensorflow`.
But what is &#39;SAME&#39; padding of max pool in `tensorflow`?


  [1]: https://arxiv.org/pdf/1603.07285v1.pdf

What is the difference between &#39;SAME&#39; and &#39;VALID&#39; padding in tf.nn.max_pool of tensorflow?

I am using Keras (with Theano) to train my CNN model. Does anyone has idea how can I use it in my C++ application? Does anyone tried something similar? I have idea to write some python code that will generate a c++ code with network functions - any suggestion on it?

I found a similar question [here][1] how to use Tensorflow Keras model in C++ but without answer.


  [1]: https://stackoverflow.com/questions/36412098/convert-keras-model-to-tensorflow-protobuf

Convert Keras model to C++

I have a simple NN model for detecting hand-written digits from a 28x28px image written in python using Keras (Theano backend):

    model0 = Sequential()

    #number of epochs to train for
    nb_epoch = 12
    #amount of data each iteration in an epoch sees
    batch_size = 128

    model0.add(Flatten(input_shape=(1, img_rows, img_cols)))
    model0.add(Dense(nb_classes))
    model0.add(Activation(&#39;softmax&#39;))
    model0.compile(loss=&#39;categorical_crossentropy&#39;, 
             optimizer=&#39;sgd&#39;,
             metrics=[&#39;accuracy&#39;])

    model0.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,
          verbose=1, validation_data=(X_test, Y_test))

    score = model0.evaluate(X_test, Y_test, verbose=0)

    print(&#39;Test score:&#39;, score[0])
    print(&#39;Test accuracy:&#39;, score[1])

This runs well and I get ~90% accuracy. I then perform the following command to get a summary of my network&#39;s structure by doing `print(model0.summary())`. This outputs the following:

    Layer (type)         Output Shape   Param #     Connected to                     
    =====================================================================
    flatten_1 (Flatten)   (None, 784)     0           flatten_input_1[0][0]            
    dense_1 (Dense)     (None, 10)       7850        flatten_1[0][0]                  
    activation_1        (None, 10)          0           dense_1[0][0]                    
    ======================================================================
    Total params: 7850

I don&#39;t understand how they get to 7850 total params and what that actually means?

Keras model.summary() result - Understanding the # of Parameters

I have a few thousand audio files and I want to classify them using Keras and Theano. So far, I generated a 28x28 spectrograms (bigger is probably better, but I am just trying to get the algorithm work at this point) of each audio file and read the image into a matrix. So in the end I get this big image matrix to feed into the network for image classification.

In a tutorial I found this mnist classification code:

    import numpy as np
    
    from keras.datasets import mnist
    from keras.models import Sequential
    from keras.layers.core import Dense
    from keras.utils import np_utils
    
    batch_size = 128
    nb_classes = 10
    nb_epochs = 2
    
    (X_train, y_train), (X_test, y_test) = mnist.load_data()
    
    X_train = X_train.reshape(60000, 784)
    X_test = X_test.reshape(10000, 784)
    X_train = X_train.astype(&quot;float32&quot;)
    X_test = X_test.astype(&quot;float32&quot;)
    X_train /= 255
    X_test /= 255
    
    print(X_train.shape[0], &quot;train samples&quot;)
    print(X_test.shape[0], &quot;test samples&quot;)
    
    y_train = np_utils.to_categorical(y_train, nb_classes)
    y_test =  np_utils.to_categorical(y_test, nb_classes)
    
    model = Sequential()
    
    model.add(Dense(output_dim = 100, input_dim = 784, activation= &quot;relu&quot;))
    model.add(Dense(output_dim = 200, activation = &quot;relu&quot;))
    model.add(Dense(output_dim = 200, activation = &quot;relu&quot;))
    model.add(Dense(output_dim = nb_classes, activation = &quot;softmax&quot;))
    
    model.compile(optimizer = &quot;adam&quot;, loss = &quot;categorical_crossentropy&quot;)
    
    model.fit(X_train, y_train, batch_size = batch_size, nb_epoch = nb_epochs, show_accuracy = True, verbose = 2, validation_data = (X_test, y_test))
    score = model.evaluate(X_test, y_test, show_accuracy = True, verbose = 0)
    print(&quot;Test score: &quot;, score[0])
    print(&quot;Test accuracy: &quot;, score[1])

This code runs, and I get the result as expected:

    (60000L, &#39;train samples&#39;)
    (10000L, &#39;test samples&#39;)
    Train on 60000 samples, validate on 10000 samples
    Epoch 1/2
    2s - loss: 0.2988 - acc: 0.9131 - val_loss: 0.1314 - val_acc: 0.9607
    Epoch 2/2
    2s - loss: 0.1144 - acc: 0.9651 - val_loss: 0.0995 - val_acc: 0.9673
    (&#39;Test score: &#39;, 0.099454972004890438)
    (&#39;Test accuracy: &#39;, 0.96730000000000005)

Up to this point everything runs perfectly, however when I apply the above algorithm to my dataset, accuracy gets stuck.

My code is as follows:

    import os
    
    import pandas as pd
    
    from sklearn.cross_validation import train_test_split
    
    from keras.models import Sequential
    from keras.layers.convolutional import Convolution2D, MaxPooling2D
    from keras.layers.core import Dense, Activation, Dropout, Flatten
    from keras.utils import np_utils
    
    import AudioProcessing as ap
    import ImageTools as it
    
    batch_size = 128
    nb_classes = 2
    nb_epoch = 10  
    
    
    for i in range(20):
        print &quot;\n&quot;
    # Generate spectrograms if necessary
    if(len(os.listdir(&quot;./AudioNormalPathalogicClassification/Image&quot;)) &gt; 0):
        print &quot;Audio files are already processed. Skipping...&quot;
    else:
        print &quot;Generating spectrograms for the audio files...&quot;
        ap.audio_2_image(&quot;./AudioNormalPathalogicClassification/Audio/&quot;,&quot;./AudioNormalPathalogicClassification/Image/&quot;,&quot;.wav&quot;,&quot;.png&quot;,(28,28))
    
    # Read the result csv
    df = pd.read_csv(&#39;./AudioNormalPathalogicClassification/Result/result.csv&#39;, header = None)
    
    df.columns = [&quot;RegionName&quot;,&quot;IsNormal&quot;]
    
    bool_mapping = {True : 1, False : 0}
    
    nb_classes = 2
    
    for col in df:
        if(col == &quot;RegionName&quot;):
            a = 3      
        else:
            df[col] = df[col].map(bool_mapping)
    
    y = df.iloc[:,1:].values
    
    y = np_utils.to_categorical(y, nb_classes)
    
    # Load images into memory
    print &quot;Loading images into memory...&quot;
    X = it.load_images(&quot;./AudioNormalPathalogicClassification/Image/&quot;,&quot;.png&quot;)
    
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 0)
    
    X_train = X_train.reshape(X_train.shape[0], 784)
    X_test = X_test.reshape(X_test.shape[0], 784)
    X_train = X_train.astype(&quot;float32&quot;)
    X_test = X_test.astype(&quot;float32&quot;)
    X_train /= 255
    X_test /= 255
    
    print(&quot;X_train shape: &quot; + str(X_train.shape))
    print(str(X_train.shape[0]) + &quot; train samples&quot;)
    print(str(X_test.shape[0]) + &quot; test samples&quot;)
    
    model = Sequential()
    
    
    model.add(Dense(output_dim = 100, input_dim = 784, activation= &quot;relu&quot;))
    model.add(Dense(output_dim = 200, activation = &quot;relu&quot;))
    model.add(Dense(output_dim = 200, activation = &quot;relu&quot;))
    model.add(Dense(output_dim = nb_classes, activation = &quot;softmax&quot;))
    
    model.compile(loss = &quot;categorical_crossentropy&quot;, optimizer = &quot;adam&quot;)
    
    print model.summary()
    
    model.fit(X_train, y_train, batch_size = batch_size, nb_epoch = nb_epoch, show_accuracy = True, verbose = 1, validation_data = (X_test, y_test))
    score = model.evaluate(X_test, y_test, show_accuracy = True, verbose = 1)
    print(&quot;Test score: &quot;, score[0])
    print(&quot;Test accuracy: &quot;, score[1])

AudioProcessing.py

    import os
    import scipy as sp
    import scipy.io.wavfile as wav
    import matplotlib.pylab as pylab
    import Image
    
    def save_spectrogram_scipy(source_filename, destination_filename, size):
        dt = 0.0005
        NFFT = 1024       
        Fs = int(1.0/dt)  
        fs, audio = wav.read(source_filename)
        if(len(audio.shape) &gt;= 2):
            audio = sp.mean(audio, axis = 1)
        fig = pylab.figure()    
        ax = pylab.Axes(fig, [0,0,1,1])    
        ax.set_axis_off()
        fig.add_axes(ax) 
        pylab.specgram(audio, NFFT = NFFT, Fs = Fs, noverlap = 900, cmap=&quot;gray&quot;)
        pylab.savefig(destination_filename)
        img = Image.open(destination_filename).convert(&quot;L&quot;)
        img = img.resize(size)
        img.save(destination_filename)
        pylab.clf()
        del img
    
    def audio_2_image(source_directory, destination_directory, audio_extension, image_extension, size):
        nb_files = len(os.listdir(source_directory));
        count = 0
        for file in os.listdir(source_directory):
            if file.endswith(audio_extension):        
                destinationName = file[:-4]
                save_spectrogram_scipy(source_directory + file, destination_directory + destinationName + image_extension, size)
                count += 1
                print (&quot;Generating spectrogram for files &quot; + str(count) + &quot; / &quot; + str(nb_files) + &quot;.&quot;)

ImageTools.py

    import os
    import numpy as np
    import matplotlib.image as mpimg
    def load_images(source_directory, image_extension):
        image_matrix = []
        nb_files = len(os.listdir(source_directory));
        count = 0
        for file in os.listdir(source_directory):
            if file.endswith(image_extension):
                with open(source_directory + file,&quot;r+b&quot;) as f:
                    img = mpimg.imread(f)
                    img = img.flatten()                
                    image_matrix.append(img)
                    del img
                    count += 1
                    #print (&quot;File &quot; + str(count) + &quot; / &quot; + str(nb_files) + &quot; loaded.&quot;)
        return np.asarray(image_matrix)

So I run the above code and recieve:

    Audio files are already processed. Skipping...
    Loading images into memory...
    X_train shape: (2394L, 784L)
    2394 train samples
    1027 test samples
    --------------------------------------------------------------------------------
    Initial input shape: (None, 784)
    --------------------------------------------------------------------------------
    Layer (name)                  Output Shape                  Param #
    --------------------------------------------------------------------------------
    Dense (dense)                 (None, 100)                   78500
    Dense (dense)                 (None, 200)                   20200
    Dense (dense)                 (None, 200)                   40200
    Dense (dense)                 (None, 2)                     402
    --------------------------------------------------------------------------------
    Total params: 139302
    --------------------------------------------------------------------------------
    None
    Train on 2394 samples, validate on 1027 samples
    Epoch 1/10
    2394/2394 [==============================] - 0s - loss: 0.6898 - acc: 0.5455 - val_loss: 0.6835 - val_acc: 0.5716
    Epoch 2/10
    2394/2394 [==============================] - 0s - loss: 0.6879 - acc: 0.5522 - val_loss: 0.6901 - val_acc: 0.5716
    Epoch 3/10
    2394/2394 [==============================] - 0s - loss: 0.6880 - acc: 0.5522 - val_loss: 0.6842 - val_acc: 0.5716
    Epoch 4/10
    2394/2394 [==============================] - 0s - loss: 0.6883 - acc: 0.5522 - val_loss: 0.6829 - val_acc: 0.5716
    Epoch 5/10
    2394/2394 [==============================] - 0s - loss: 0.6885 - acc: 0.5522 - val_loss: 0.6836 - val_acc: 0.5716
    Epoch 6/10
    2394/2394 [==============================] - 0s - loss: 0.6887 - acc: 0.5522 - val_loss: 0.6832 - val_acc: 0.5716
    Epoch 7/10
    2394/2394 [==============================] - 0s - loss: 0.6882 - acc: 0.5522 - val_loss: 0.6859 - val_acc: 0.5716
    Epoch 8/10
    2394/2394 [==============================] - 0s - loss: 0.6882 - acc: 0.5522 - val_loss: 0.6849 - val_acc: 0.5716
    Epoch 9/10
    2394/2394 [==============================] - 0s - loss: 0.6885 - acc: 0.5522 - val_loss: 0.6836 - val_acc: 0.5716
    Epoch 10/10
    2394/2394 [==============================] - 0s - loss: 0.6877 - acc: 0.5522 - val_loss: 0.6849 - val_acc: 0.5716
    1027/1027 [==============================] - 0s
    (&#39;Test score: &#39;, 0.68490593621422047)
    (&#39;Test accuracy: &#39;, 0.57156767283349563)

I tried changing the network, adding more epochs, but I always get the same result no matter what. I don&#39;t understand why I am getting the same result.

Any help would be appreciated. Thank you.

Edit:
I found a mistake where pixel values were not read correctly. I fixed the ImageTools.py below as:

    import os
    import numpy as np
    from scipy.misc import imread
    
    def load_images(source_directory, image_extension):
        image_matrix = []
        nb_files = len(os.listdir(source_directory));
        count = 0
        for file in os.listdir(source_directory):
            if file.endswith(image_extension):
                with open(source_directory + file,&quot;r+b&quot;) as f:
                    img = imread(f)                
                    img = img.flatten()                        
                    image_matrix.append(img)
                    del img
                    count += 1
                    #print (&quot;File &quot; + str(count) + &quot; / &quot; + str(nb_files) + &quot; loaded.&quot;)
        return np.asarray(image_matrix)

Now I actually get grayscale pixel values from 0 to 255, so now my dividing it by 255 makes sense. However, I still get the same result.

Content Type	Original Author	Original Content on Stackoverflow
Question	Anuj Gupta	View Question on Stackoverflow
Solution 1 - Deep Learning	Serj Zaharchenko	View Answer on Stackoverflow
Solution 2 - Deep Learning	Anuj Gupta	View Answer on Stackoverflow
Solution 3 - Deep Learning	Miguel A. Sanchez-Perez	View Answer on Stackoverflow
Solution 4 - Deep Learning	Eaton	View Answer on Stackoverflow

How to find Number of parameters of a keras model?

Deep Learning Problem Overview

Deep Learning Solutions

Solution 1 - Deep Learning

Solution 2 - Deep Learning

Solution 3 - Deep Learning

Solution 4 - Deep Learning

Bulk update mysql with where statement

Template literals with nested backticks(`) in ES6

Attributions