Why model performs poor after normalization?
I'm using fully connected neural network and I am using normalized data such that every single sample values range from 0 to 1. I have used 100 neurons in first layer and 10 in second layer and used almost 50 lack samples during training. I want to classify my data into two classes. But my networks performance is too low, almost 49 percent on training and test data. I tried to increase the performance by changing the values of hyper parameters. But it didn't work. Can some one please tell me what should I do to get higher performance?
x = tf.placeholder(tf.float32, [None, nPixels])
W1 = tf.Variable(tf.random_normal([nPixels, nNodes1], stddev=0.01))
b1 = tf.Variable(tf.zeros([nNodes1]))
y1 = tf.nn.relu(tf.matmul(x, W1) + b1)
W2 = tf.Variable(tf.random_normal([nNodes1, nNodes2], stddev=0.01))
b2 = tf.Variable(tf.zeros([nNodes2]))
y2 = tf.nn.relu(tf.matmul(y1, W2) + b2)
W3 = tf.Variable(tf.random_normal([nNodes2, nLabels], stddev=0.01))
b3 = tf.Variable(tf.zeros([nLabels]))
y = tf.nn.softmax(tf.matmul(y2, W3) + b3)
y_ = tf.placeholder(dtype=tf.float32, shape=[None, 2])
cross_entropy = -1*tf.reduce_sum(y_* tf.log(y), axis=1)
loss = tf.reduce_mean(cross_entropy)
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
correct_prediction = tf.equal(tf.argmax(y_,axis=1), tf.argmax(y, axis=1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
python tensorflow machine-learning neural-network
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
asked Nov 12 '18 at 3:41
R.joe
46
|
show 2 more comments
I'm using fully connected neural network and I am using normalized data such that every single sample values range from 0 to 1. I have used 100 neurons in first layer and 10 in second layer and used almost 50 lack samples during training. I want to classify my data into two classes. But my networks performance is too low, almost 49 percent on training and test data. I tried to increase the performance by changing the values of hyper parameters. But it didn't work. Can some one please tell me what should I do to get higher performance?
x = tf.placeholder(tf.float32, [None, nPixels])
W1 = tf.Variable(tf.random_normal([nPixels, nNodes1], stddev=0.01))
b1 = tf.Variable(tf.zeros([nNodes1]))
y1 = tf.nn.relu(tf.matmul(x, W1) + b1)
W2 = tf.Variable(tf.random_normal([nNodes1, nNodes2], stddev=0.01))
b2 = tf.Variable(tf.zeros([nNodes2]))
y2 = tf.nn.relu(tf.matmul(y1, W2) + b2)
W3 = tf.Variable(tf.random_normal([nNodes2, nLabels], stddev=0.01))
b3 = tf.Variable(tf.zeros([nLabels]))
y = tf.nn.softmax(tf.matmul(y2, W3) + b3)
y_ = tf.placeholder(dtype=tf.float32, shape=[None, 2])
cross_entropy = -1*tf.reduce_sum(y_* tf.log(y), axis=1)
loss = tf.reduce_mean(cross_entropy)
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
correct_prediction = tf.equal(tf.argmax(y_,axis=1), tf.argmax(y, axis=1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
python tensorflow machine-learning neural-network
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
asked Nov 12 '18 at 3:41
R.joe
46
The title of your question suggests that it was working before you did 'normalization'. Could you be specific on what has changed?
– dedObed
Nov 12 '18 at 7:33
Yes my model was giving 98 percent accuracy without normalization. I have performed normalization on the input data having shape (samples,features). I have normalized the feature of every sample such that in each single sample feature ranges from 0 to one. This is the only change that I have made in input data. After this accuracy suddenly drops. I want to improve performance after normalization
– R.joe
Nov 12 '18 at 7:49
What is the variance of your data?
– Novak
Nov 12 '18 at 8:14
I didn't check the variance. Should I check it for each sample? Why we nned variance if I normalize each sample from 0 to 1?
– R.joe
Nov 12 '18 at 8:17
If the performance was 98% anyway, why would you change anything?
– cheersmate
Nov 12 '18 at 8:17
|
show 2 more comments
I'm using fully connected neural network and I am using normalized data such that every single sample values range from 0 to 1. I have used 100 neurons in first layer and 10 in second layer and used almost 50 lack samples during training. I want to classify my data into two classes. But my networks performance is too low, almost 49 percent on training and test data. I tried to increase the performance by changing the values of hyper parameters. But it didn't work. Can some one please tell me what should I do to get higher performance?
x = tf.placeholder(tf.float32, [None, nPixels])
W1 = tf.Variable(tf.random_normal([nPixels, nNodes1], stddev=0.01))
b1 = tf.Variable(tf.zeros([nNodes1]))
y1 = tf.nn.relu(tf.matmul(x, W1) + b1)
W2 = tf.Variable(tf.random_normal([nNodes1, nNodes2], stddev=0.01))
b2 = tf.Variable(tf.zeros([nNodes2]))
y2 = tf.nn.relu(tf.matmul(y1, W2) + b2)
W3 = tf.Variable(tf.random_normal([nNodes2, nLabels], stddev=0.01))
b3 = tf.Variable(tf.zeros([nLabels]))
y = tf.nn.softmax(tf.matmul(y2, W3) + b3)
y_ = tf.placeholder(dtype=tf.float32, shape=[None, 2])
cross_entropy = -1*tf.reduce_sum(y_* tf.log(y), axis=1)
loss = tf.reduce_mean(cross_entropy)
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
correct_prediction = tf.equal(tf.argmax(y_,axis=1), tf.argmax(y, axis=1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
python tensorflow machine-learning neural-network
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
asked Nov 12 '18 at 3:41
R.joe
46
I'm using fully connected neural network and I am using normalized data such that every single sample values range from 0 to 1. I have used 100 neurons in first layer and 10 in second layer and used almost 50 lack samples during training. I want to classify my data into two classes. But my networks performance is too low, almost 49 percent on training and test data. I tried to increase the performance by changing the values of hyper parameters. But it didn't work. Can some one please tell me what should I do to get higher performance?
x = tf.placeholder(tf.float32, [None, nPixels])
W1 = tf.Variable(tf.random_normal([nPixels, nNodes1], stddev=0.01))
b1 = tf.Variable(tf.zeros([nNodes1]))
y1 = tf.nn.relu(tf.matmul(x, W1) + b1)
W2 = tf.Variable(tf.random_normal([nNodes1, nNodes2], stddev=0.01))
b2 = tf.Variable(tf.zeros([nNodes2]))
y2 = tf.nn.relu(tf.matmul(y1, W2) + b2)
W3 = tf.Variable(tf.random_normal([nNodes2, nLabels], stddev=0.01))
b3 = tf.Variable(tf.zeros([nLabels]))
y = tf.nn.softmax(tf.matmul(y2, W3) + b3)
y_ = tf.placeholder(dtype=tf.float32, shape=[None, 2])
cross_entropy = -1*tf.reduce_sum(y_* tf.log(y), axis=1)
loss = tf.reduce_mean(cross_entropy)
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
correct_prediction = tf.equal(tf.argmax(y_,axis=1), tf.argmax(y, axis=1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
python tensorflow machine-learning neural-network
python tensorflow machine-learning neural-network
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
asked Nov 12 '18 at 3:41
R.joe
46
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
asked Nov 12 '18 at 3:41
R.joe
46
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
edited Nov 12 '18 at 6:51
Milo Lu
1,59911327
1,59911327
asked Nov 12 '18 at 3:41
R.joe
46
asked Nov 12 '18 at 3:41
R.joe
46
asked Nov 12 '18 at 3:41
R.joe
46
46
The title of your question suggests that it was working before you did 'normalization'. Could you be specific on what has changed?
– dedObed
Nov 12 '18 at 7:33
Yes my model was giving 98 percent accuracy without normalization. I have performed normalization on the input data having shape (samples,features). I have normalized the feature of every sample such that in each single sample feature ranges from 0 to one. This is the only change that I have made in input data. After this accuracy suddenly drops. I want to improve performance after normalization
– R.joe
Nov 12 '18 at 7:49
What is the variance of your data?
– Novak
Nov 12 '18 at 8:14
I didn't check the variance. Should I check it for each sample? Why we nned variance if I normalize each sample from 0 to 1?
– R.joe
Nov 12 '18 at 8:17
If the performance was 98% anyway, why would you change anything?
– cheersmate
Nov 12 '18 at 8:17
|
show 2 more comments
The title of your question suggests that it was working before you did 'normalization'. Could you be specific on what has changed?
– dedObed
Nov 12 '18 at 7:33
Yes my model was giving 98 percent accuracy without normalization. I have performed normalization on the input data having shape (samples,features). I have normalized the feature of every sample such that in each single sample feature ranges from 0 to one. This is the only change that I have made in input data. After this accuracy suddenly drops. I want to improve performance after normalization
– R.joe
Nov 12 '18 at 7:49
What is the variance of your data?
– Novak
Nov 12 '18 at 8:14
I didn't check the variance. Should I check it for each sample? Why we nned variance if I normalize each sample from 0 to 1?
– R.joe
Nov 12 '18 at 8:17
If the performance was 98% anyway, why would you change anything?
– cheersmate
Nov 12 '18 at 8:17
The title of your question suggests that it was working before you did 'normalization'. Could you be specific on what has changed?
– dedObed
Nov 12 '18 at 7:33
The title of your question suggests that it was working before you did 'normalization'. Could you be specific on what has changed?
– dedObed
Nov 12 '18 at 7:33
Yes my model was giving 98 percent accuracy without normalization. I have performed normalization on the input data having shape (samples,features). I have normalized the feature of every sample such that in each single sample feature ranges from 0 to one. This is the only change that I have made in input data. After this accuracy suddenly drops. I want to improve performance after normalization
– R.joe
Nov 12 '18 at 7:49
Yes my model was giving 98 percent accuracy without normalization. I have performed normalization on the input data having shape (samples,features). I have normalized the feature of every sample such that in each single sample feature ranges from 0 to one. This is the only change that I have made in input data. After this accuracy suddenly drops. I want to improve performance after normalization
– R.joe
Nov 12 '18 at 7:49
What is the variance of your data?
– Novak
Nov 12 '18 at 8:14
What is the variance of your data?
– Novak
Nov 12 '18 at 8:14
I didn't check the variance. Should I check it for each sample? Why we nned variance if I normalize each sample from 0 to 1?
– R.joe
Nov 12 '18 at 8:17
I didn't check the variance. Should I check it for each sample? Why we nned variance if I normalize each sample from 0 to 1?
– R.joe
Nov 12 '18 at 8:17
If the performance was 98% anyway, why would you change anything?
– cheersmate
Nov 12 '18 at 8:17
If the performance was 98% anyway, why would you change anything?
– cheersmate
Nov 12 '18 at 8:17
|
show 2 more comments
1 Answer
1
active
oldest
votes
Your computational model knows nothing about "images", it only sees numbers. So if you trained it with pixels of values from 0-255, it has learned what "light" means, what "dark" means and how do these combine to give you whatever target value you try model.
And what you did by the normalization is that you forced all pixel to be 0-1. So as far as the model cares, they are all black as night. No surprise that it cannot extract anything meaningful.
You need to apply the same input normalization during both training and testing.
And speaking about normalization for NN models, it is better to normalize to zero mean.
answered Nov 12 '18 at 8:45
dedObed
419110
I have applied this normalization to both training and test set. Suppose my input values in one sample were -4 to +4 so by normalization I shifted them on a different scale 0 to 1. Why should I expect that machine can not extract any information. I even tried to make model complex by adding different layers and provided more data but still there is no improvement in accuracy. I'm bound to implement min max normalization.
– R.joe
Nov 12 '18 at 10:08
But have you trained the model with the normalized values? From reading your comment under the question, I got the idea that you have not.
– dedObed
Nov 12 '18 at 10:09
Yes I trained the model with normalized values. Above code that I have mentioned contains two hidden layer. I varied number of neurons ,and also added a large amount of data. But still performance is too low. I was wondering how to improve it
– R.joe
Nov 12 '18 at 10:16
Hey, that changes things, you should have said that ;-) But really, try to normalize them to zero mean, e.g. to <-1; 1> (and check the distribution, there may be outliers screwing the min-max normalization heavily). Also, for such a small network,tanhcan be expected to work better thanrelu. And finally, learning rate tuning typically gives you the most. Try lowering it geometrically. Do changes one by one.
– dedObed
Nov 12 '18 at 10:25
if i normalize b/w -1 and 1 is is similar to 0 to 1 means shifting a scale. Why should I expect that in this case network will perform better?
– R.joe
Nov 12 '18 at 11:17
|
show 3 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53255677%2fwhy-model-performs-poor-after-normalization%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Your computational model knows nothing about "images", it only sees numbers. So if you trained it with pixels of values from 0-255, it has learned what "light" means, what "dark" means and how do these combine to give you whatever target value you try model.
And what you did by the normalization is that you forced all pixel to be 0-1. So as far as the model cares, they are all black as night. No surprise that it cannot extract anything meaningful.
You need to apply the same input normalization during both training and testing.
And speaking about normalization for NN models, it is better to normalize to zero mean.
answered Nov 12 '18 at 8:45
dedObed
419110
I have applied this normalization to both training and test set. Suppose my input values in one sample were -4 to +4 so by normalization I shifted them on a different scale 0 to 1. Why should I expect that machine can not extract any information. I even tried to make model complex by adding different layers and provided more data but still there is no improvement in accuracy. I'm bound to implement min max normalization.
– R.joe
Nov 12 '18 at 10:08
But have you trained the model with the normalized values? From reading your comment under the question, I got the idea that you have not.
– dedObed
Nov 12 '18 at 10:09
Yes I trained the model with normalized values. Above code that I have mentioned contains two hidden layer. I varied number of neurons ,and also added a large amount of data. But still performance is too low. I was wondering how to improve it
– R.joe
Nov 12 '18 at 10:16
Hey, that changes things, you should have said that ;-) But really, try to normalize them to zero mean, e.g. to <-1; 1> (and check the distribution, there may be outliers screwing the min-max normalization heavily). Also, for such a small network,tanhcan be expected to work better thanrelu. And finally, learning rate tuning typically gives you the most. Try lowering it geometrically. Do changes one by one.
– dedObed
Nov 12 '18 at 10:25
if i normalize b/w -1 and 1 is is similar to 0 to 1 means shifting a scale. Why should I expect that in this case network will perform better?
– R.joe
Nov 12 '18 at 11:17
|
show 3 more comments
Your computational model knows nothing about "images", it only sees numbers. So if you trained it with pixels of values from 0-255, it has learned what "light" means, what "dark" means and how do these combine to give you whatever target value you try model.
And what you did by the normalization is that you forced all pixel to be 0-1. So as far as the model cares, they are all black as night. No surprise that it cannot extract anything meaningful.
You need to apply the same input normalization during both training and testing.
And speaking about normalization for NN models, it is better to normalize to zero mean.
answered Nov 12 '18 at 8:45
dedObed
419110
I have applied this normalization to both training and test set. Suppose my input values in one sample were -4 to +4 so by normalization I shifted them on a different scale 0 to 1. Why should I expect that machine can not extract any information. I even tried to make model complex by adding different layers and provided more data but still there is no improvement in accuracy. I'm bound to implement min max normalization.
– R.joe
Nov 12 '18 at 10:08
But have you trained the model with the normalized values? From reading your comment under the question, I got the idea that you have not.
– dedObed
Nov 12 '18 at 10:09
Yes I trained the model with normalized values. Above code that I have mentioned contains two hidden layer. I varied number of neurons ,and also added a large amount of data. But still performance is too low. I was wondering how to improve it
– R.joe
Nov 12 '18 at 10:16
Hey, that changes things, you should have said that ;-) But really, try to normalize them to zero mean, e.g. to <-1; 1> (and check the distribution, there may be outliers screwing the min-max normalization heavily). Also, for such a small network,tanhcan be expected to work better thanrelu. And finally, learning rate tuning typically gives you the most. Try lowering it geometrically. Do changes one by one.
– dedObed
Nov 12 '18 at 10:25
if i normalize b/w -1 and 1 is is similar to 0 to 1 means shifting a scale. Why should I expect that in this case network will perform better?
– R.joe
Nov 12 '18 at 11:17
|
show 3 more comments
Your computational model knows nothing about "images", it only sees numbers. So if you trained it with pixels of values from 0-255, it has learned what "light" means, what "dark" means and how do these combine to give you whatever target value you try model.
And what you did by the normalization is that you forced all pixel to be 0-1. So as far as the model cares, they are all black as night. No surprise that it cannot extract anything meaningful.
You need to apply the same input normalization during both training and testing.
And speaking about normalization for NN models, it is better to normalize to zero mean.
answered Nov 12 '18 at 8:45
dedObed
419110
Your computational model knows nothing about "images", it only sees numbers. So if you trained it with pixels of values from 0-255, it has learned what "light" means, what "dark" means and how do these combine to give you whatever target value you try model.
And what you did by the normalization is that you forced all pixel to be 0-1. So as far as the model cares, they are all black as night. No surprise that it cannot extract anything meaningful.
You need to apply the same input normalization during both training and testing.
And speaking about normalization for NN models, it is better to normalize to zero mean.
answered Nov 12 '18 at 8:45
dedObed
419110
answered Nov 12 '18 at 8:45
dedObed
419110
answered Nov 12 '18 at 8:45
dedObed
419110
answered Nov 12 '18 at 8:45
dedObed
419110
419110
I have applied this normalization to both training and test set. Suppose my input values in one sample were -4 to +4 so by normalization I shifted them on a different scale 0 to 1. Why should I expect that machine can not extract any information. I even tried to make model complex by adding different layers and provided more data but still there is no improvement in accuracy. I'm bound to implement min max normalization.
– R.joe
Nov 12 '18 at 10:08
But have you trained the model with the normalized values? From reading your comment under the question, I got the idea that you have not.
– dedObed
Nov 12 '18 at 10:09
Yes I trained the model with normalized values. Above code that I have mentioned contains two hidden layer. I varied number of neurons ,and also added a large amount of data. But still performance is too low. I was wondering how to improve it
– R.joe
Nov 12 '18 at 10:16
Hey, that changes things, you should have said that ;-) But really, try to normalize them to zero mean, e.g. to <-1; 1> (and check the distribution, there may be outliers screwing the min-max normalization heavily). Also, for such a small network,tanhcan be expected to work better thanrelu. And finally, learning rate tuning typically gives you the most. Try lowering it geometrically. Do changes one by one.
– dedObed
Nov 12 '18 at 10:25
if i normalize b/w -1 and 1 is is similar to 0 to 1 means shifting a scale. Why should I expect that in this case network will perform better?
– R.joe
Nov 12 '18 at 11:17
|
show 3 more comments
I have applied this normalization to both training and test set. Suppose my input values in one sample were -4 to +4 so by normalization I shifted them on a different scale 0 to 1. Why should I expect that machine can not extract any information. I even tried to make model complex by adding different layers and provided more data but still there is no improvement in accuracy. I'm bound to implement min max normalization.
– R.joe
Nov 12 '18 at 10:08
But have you trained the model with the normalized values? From reading your comment under the question, I got the idea that you have not.
– dedObed
Nov 12 '18 at 10:09
Yes I trained the model with normalized values. Above code that I have mentioned contains two hidden layer. I varied number of neurons ,and also added a large amount of data. But still performance is too low. I was wondering how to improve it
– R.joe
Nov 12 '18 at 10:16
Hey, that changes things, you should have said that ;-) But really, try to normalize them to zero mean, e.g. to <-1; 1> (and check the distribution, there may be outliers screwing the min-max normalization heavily). Also, for such a small network,tanhcan be expected to work better thanrelu. And finally, learning rate tuning typically gives you the most. Try lowering it geometrically. Do changes one by one.
– dedObed
Nov 12 '18 at 10:25
if i normalize b/w -1 and 1 is is similar to 0 to 1 means shifting a scale. Why should I expect that in this case network will perform better?
– R.joe
Nov 12 '18 at 11:17
I have applied this normalization to both training and test set. Suppose my input values in one sample were -4 to +4 so by normalization I shifted them on a different scale 0 to 1. Why should I expect that machine can not extract any information. I even tried to make model complex by adding different layers and provided more data but still there is no improvement in accuracy. I'm bound to implement min max normalization.
– R.joe
Nov 12 '18 at 10:08
I have applied this normalization to both training and test set. Suppose my input values in one sample were -4 to +4 so by normalization I shifted them on a different scale 0 to 1. Why should I expect that machine can not extract any information. I even tried to make model complex by adding different layers and provided more data but still there is no improvement in accuracy. I'm bound to implement min max normalization.
– R.joe
Nov 12 '18 at 10:08
But have you trained the model with the normalized values? From reading your comment under the question, I got the idea that you have not.
– dedObed
Nov 12 '18 at 10:09
But have you trained the model with the normalized values? From reading your comment under the question, I got the idea that you have not.
– dedObed
Nov 12 '18 at 10:09
Yes I trained the model with normalized values. Above code that I have mentioned contains two hidden layer. I varied number of neurons ,and also added a large amount of data. But still performance is too low. I was wondering how to improve it
– R.joe
Nov 12 '18 at 10:16
Yes I trained the model with normalized values. Above code that I have mentioned contains two hidden layer. I varied number of neurons ,and also added a large amount of data. But still performance is too low. I was wondering how to improve it
– R.joe
Nov 12 '18 at 10:16
Hey, that changes things, you should have said that ;-) But really, try to normalize them to zero mean, e.g. to <-1; 1> (and check the distribution, there may be outliers screwing the min-max normalization heavily). Also, for such a small network,
tanh can be expected to work better than relu. And finally, learning rate tuning typically gives you the most. Try lowering it geometrically. Do changes one by one.– dedObed
Nov 12 '18 at 10:25
Hey, that changes things, you should have said that ;-) But really, try to normalize them to zero mean, e.g. to <-1; 1> (and check the distribution, there may be outliers screwing the min-max normalization heavily). Also, for such a small network,
tanh can be expected to work better than relu. And finally, learning rate tuning typically gives you the most. Try lowering it geometrically. Do changes one by one.– dedObed
Nov 12 '18 at 10:25
if i normalize b/w -1 and 1 is is similar to 0 to 1 means shifting a scale. Why should I expect that in this case network will perform better?
– R.joe
Nov 12 '18 at 11:17
if i normalize b/w -1 and 1 is is similar to 0 to 1 means shifting a scale. Why should I expect that in this case network will perform better?
– R.joe
Nov 12 '18 at 11:17
|
show 3 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53255677%2fwhy-model-performs-poor-after-normalization%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
The title of your question suggests that it was working before you did 'normalization'. Could you be specific on what has changed?
– dedObed
Nov 12 '18 at 7:33
Yes my model was giving 98 percent accuracy without normalization. I have performed normalization on the input data having shape (samples,features). I have normalized the feature of every sample such that in each single sample feature ranges from 0 to one. This is the only change that I have made in input data. After this accuracy suddenly drops. I want to improve performance after normalization
– R.joe
Nov 12 '18 at 7:49
What is the variance of your data?
– Novak
Nov 12 '18 at 8:14
I didn't check the variance. Should I check it for each sample? Why we nned variance if I normalize each sample from 0 to 1?
– R.joe
Nov 12 '18 at 8:17
If the performance was 98% anyway, why would you change anything?
– cheersmate
Nov 12 '18 at 8:17