how to create sparseVector with index and value in Tensorflow, python?












0














i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1









share|improve this question






















  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 '18 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 '18 at 7:40
















0














i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1









share|improve this question






















  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 '18 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 '18 at 7:40














0












0








0







i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1









share|improve this question













i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1






python tensorflow vector






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 12 '18 at 13:01









hvchyshvchys

2127




2127












  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 '18 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 '18 at 7:40


















  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 '18 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 '18 at 7:40
















Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07




Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 '18 at 14:07












thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40




thx, problem solved, I post an answer
– hvchys
Nov 13 '18 at 7:40












1 Answer
1






active

oldest

votes


















0














problem solved



import tensorflow as tf
from tensorflow import TensorShape, Dimension


class GetVector:

@classmethod
def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
batch_size = idx_all_0.shape[0].value
'''
cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
cur_val = tf.placeholder(tf.float64, [batch_size, None])
cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
'''
idx_all = cls.idx_reform(idx_all_0)

ans =
for i in range(batch_size):
cur_idx = idx_all[i]
cur_val = val_all[i]
cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
ans = tf.concat([ans, cur_vec_tensor], 0)
ans = tf.reshape(ans, [batch_size, dim_num])
return ans

@classmethod
def idx_reform(cls, idx_all):
batch_size = idx_all.shape[0].value
return tf.reshape(idx_all, [batch_size, -1, 1])





share|improve this answer





















    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262744%2fhow-to-create-sparsevector-with-index-and-value-in-tensorflow-python%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    problem solved



    import tensorflow as tf
    from tensorflow import TensorShape, Dimension


    class GetVector:

    @classmethod
    def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
    batch_size = idx_all_0.shape[0].value
    '''
    cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
    cur_val = tf.placeholder(tf.float64, [batch_size, None])
    cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
    '''
    idx_all = cls.idx_reform(idx_all_0)

    ans =
    for i in range(batch_size):
    cur_idx = idx_all[i]
    cur_val = val_all[i]
    cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
    cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
    ans = tf.concat([ans, cur_vec_tensor], 0)
    ans = tf.reshape(ans, [batch_size, dim_num])
    return ans

    @classmethod
    def idx_reform(cls, idx_all):
    batch_size = idx_all.shape[0].value
    return tf.reshape(idx_all, [batch_size, -1, 1])





    share|improve this answer


























      0














      problem solved



      import tensorflow as tf
      from tensorflow import TensorShape, Dimension


      class GetVector:

      @classmethod
      def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
      batch_size = idx_all_0.shape[0].value
      '''
      cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
      cur_val = tf.placeholder(tf.float64, [batch_size, None])
      cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
      '''
      idx_all = cls.idx_reform(idx_all_0)

      ans =
      for i in range(batch_size):
      cur_idx = idx_all[i]
      cur_val = val_all[i]
      cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
      cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
      ans = tf.concat([ans, cur_vec_tensor], 0)
      ans = tf.reshape(ans, [batch_size, dim_num])
      return ans

      @classmethod
      def idx_reform(cls, idx_all):
      batch_size = idx_all.shape[0].value
      return tf.reshape(idx_all, [batch_size, -1, 1])





      share|improve this answer
























        0












        0








        0






        problem solved



        import tensorflow as tf
        from tensorflow import TensorShape, Dimension


        class GetVector:

        @classmethod
        def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
        batch_size = idx_all_0.shape[0].value
        '''
        cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
        cur_val = tf.placeholder(tf.float64, [batch_size, None])
        cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
        '''
        idx_all = cls.idx_reform(idx_all_0)

        ans =
        for i in range(batch_size):
        cur_idx = idx_all[i]
        cur_val = val_all[i]
        cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
        cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
        ans = tf.concat([ans, cur_vec_tensor], 0)
        ans = tf.reshape(ans, [batch_size, dim_num])
        return ans

        @classmethod
        def idx_reform(cls, idx_all):
        batch_size = idx_all.shape[0].value
        return tf.reshape(idx_all, [batch_size, -1, 1])





        share|improve this answer












        problem solved



        import tensorflow as tf
        from tensorflow import TensorShape, Dimension


        class GetVector:

        @classmethod
        def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
        batch_size = idx_all_0.shape[0].value
        '''
        cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
        cur_val = tf.placeholder(tf.float64, [batch_size, None])
        cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
        '''
        idx_all = cls.idx_reform(idx_all_0)

        ans =
        for i in range(batch_size):
        cur_idx = idx_all[i]
        cur_val = val_all[i]
        cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
        cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
        ans = tf.concat([ans, cur_vec_tensor], 0)
        ans = tf.reshape(ans, [batch_size, dim_num])
        return ans

        @classmethod
        def idx_reform(cls, idx_all):
        batch_size = idx_all.shape[0].value
        return tf.reshape(idx_all, [batch_size, -1, 1])






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 13 '18 at 7:40









        hvchyshvchys

        2127




        2127






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262744%2fhow-to-create-sparsevector-with-index-and-value-in-tensorflow-python%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Full-time equivalent

            Bicuculline

            さくらももこ