how to create sparseVector with index and value in Tensorflow, python?
i have two Tensors, like this:
>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>
How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.
I tried this:
xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
but here comes the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1
python tensorflow vector
add a comment |
i have two Tensors, like this:
>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>
How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.
I tried this:
xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
but here comes the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1
python tensorflow vector
Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07
thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40
add a comment |
i have two Tensors, like this:
>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>
How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.
I tried this:
xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
but here comes the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1
python tensorflow vector
i have two Tensors, like this:
>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>
How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.
I tried this:
xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
but here comes the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1
python tensorflow vector
python tensorflow vector
asked Nov 12 '18 at 13:01
hvchyshvchys
2127
2127
Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07
thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40
add a comment |
Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07
thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40
Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07
Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07
thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40
thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40
add a comment |
1 Answer
1
active
oldest
votes
problem solved
import tensorflow as tf
from tensorflow import TensorShape, Dimension
class GetVector:
@classmethod
def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
batch_size = idx_all_0.shape[0].value
'''
cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
cur_val = tf.placeholder(tf.float64, [batch_size, None])
cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
'''
idx_all = cls.idx_reform(idx_all_0)
ans =
for i in range(batch_size):
cur_idx = idx_all[i]
cur_val = val_all[i]
cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
ans = tf.concat([ans, cur_vec_tensor], 0)
ans = tf.reshape(ans, [batch_size, dim_num])
return ans
@classmethod
def idx_reform(cls, idx_all):
batch_size = idx_all.shape[0].value
return tf.reshape(idx_all, [batch_size, -1, 1])
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262744%2fhow-to-create-sparsevector-with-index-and-value-in-tensorflow-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
problem solved
import tensorflow as tf
from tensorflow import TensorShape, Dimension
class GetVector:
@classmethod
def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
batch_size = idx_all_0.shape[0].value
'''
cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
cur_val = tf.placeholder(tf.float64, [batch_size, None])
cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
'''
idx_all = cls.idx_reform(idx_all_0)
ans =
for i in range(batch_size):
cur_idx = idx_all[i]
cur_val = val_all[i]
cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
ans = tf.concat([ans, cur_vec_tensor], 0)
ans = tf.reshape(ans, [batch_size, dim_num])
return ans
@classmethod
def idx_reform(cls, idx_all):
batch_size = idx_all.shape[0].value
return tf.reshape(idx_all, [batch_size, -1, 1])
add a comment |
problem solved
import tensorflow as tf
from tensorflow import TensorShape, Dimension
class GetVector:
@classmethod
def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
batch_size = idx_all_0.shape[0].value
'''
cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
cur_val = tf.placeholder(tf.float64, [batch_size, None])
cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
'''
idx_all = cls.idx_reform(idx_all_0)
ans =
for i in range(batch_size):
cur_idx = idx_all[i]
cur_val = val_all[i]
cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
ans = tf.concat([ans, cur_vec_tensor], 0)
ans = tf.reshape(ans, [batch_size, dim_num])
return ans
@classmethod
def idx_reform(cls, idx_all):
batch_size = idx_all.shape[0].value
return tf.reshape(idx_all, [batch_size, -1, 1])
add a comment |
problem solved
import tensorflow as tf
from tensorflow import TensorShape, Dimension
class GetVector:
@classmethod
def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
batch_size = idx_all_0.shape[0].value
'''
cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
cur_val = tf.placeholder(tf.float64, [batch_size, None])
cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
'''
idx_all = cls.idx_reform(idx_all_0)
ans =
for i in range(batch_size):
cur_idx = idx_all[i]
cur_val = val_all[i]
cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
ans = tf.concat([ans, cur_vec_tensor], 0)
ans = tf.reshape(ans, [batch_size, dim_num])
return ans
@classmethod
def idx_reform(cls, idx_all):
batch_size = idx_all.shape[0].value
return tf.reshape(idx_all, [batch_size, -1, 1])
problem solved
import tensorflow as tf
from tensorflow import TensorShape, Dimension
class GetVector:
@classmethod
def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
batch_size = idx_all_0.shape[0].value
'''
cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
cur_val = tf.placeholder(tf.float64, [batch_size, None])
cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
'''
idx_all = cls.idx_reform(idx_all_0)
ans =
for i in range(batch_size):
cur_idx = idx_all[i]
cur_val = val_all[i]
cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
ans = tf.concat([ans, cur_vec_tensor], 0)
ans = tf.reshape(ans, [batch_size, dim_num])
return ans
@classmethod
def idx_reform(cls, idx_all):
batch_size = idx_all.shape[0].value
return tf.reshape(idx_all, [batch_size, -1, 1])
answered Nov 13 '18 at 7:40
hvchyshvchys
2127
2127
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262744%2fhow-to-create-sparsevector-with-index-and-value-in-tensorflow-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07
thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40