reading structured binary data in python3.6 using struct











up vote
0
down vote

favorite












I really tried to look for this doubt in many ways but maybe since I never worked with binary files before I don't have any idea what the keywords to search similar things to help me out. That's why I am asking here.



So, I have a file:



path = 'myPath/file.pd0'
in_file = open(path, "rb")
read_file = in_file.read()
type(read_file)


when I try to check what is inside read_file I get:



b'x7fx7fxccx05x00x0f$x00`x00xa2x00$x02xe6x02xa8x03xd0x032x04dx04x96x04xa6x04xe0x04'


The type of read_file is bytes. When I try to use struct since it is the function people suggest I get the following error:



import struct
struct.unpack('hhl', read_file[0:30])

error: unpack requires a buffer of 16 bytes


No matter what fmt I get unpack requires a buffer of n bytes.



The file structure that I am trying to read is defined as follow:



HEADER
(6 BYTES + [2 x No. OF DATA TYPES])



FIXED LEADER DATA
(60 BYTES)



VARIABLE LEADER DATA
(66 BYTES)



CORRELATION MAGNITUDE
(2 BYTES + 4 BYTES PER DEPTH CELL)



Any idea how I could start reading these bytes using struct or something similar in python?



Thank you










share|improve this question


























    up vote
    0
    down vote

    favorite












    I really tried to look for this doubt in many ways but maybe since I never worked with binary files before I don't have any idea what the keywords to search similar things to help me out. That's why I am asking here.



    So, I have a file:



    path = 'myPath/file.pd0'
    in_file = open(path, "rb")
    read_file = in_file.read()
    type(read_file)


    when I try to check what is inside read_file I get:



    b'x7fx7fxccx05x00x0f$x00`x00xa2x00$x02xe6x02xa8x03xd0x032x04dx04x96x04xa6x04xe0x04'


    The type of read_file is bytes. When I try to use struct since it is the function people suggest I get the following error:



    import struct
    struct.unpack('hhl', read_file[0:30])

    error: unpack requires a buffer of 16 bytes


    No matter what fmt I get unpack requires a buffer of n bytes.



    The file structure that I am trying to read is defined as follow:



    HEADER
    (6 BYTES + [2 x No. OF DATA TYPES])



    FIXED LEADER DATA
    (60 BYTES)



    VARIABLE LEADER DATA
    (66 BYTES)



    CORRELATION MAGNITUDE
    (2 BYTES + 4 BYTES PER DEPTH CELL)



    Any idea how I could start reading these bytes using struct or something similar in python?



    Thank you










    share|improve this question
























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I really tried to look for this doubt in many ways but maybe since I never worked with binary files before I don't have any idea what the keywords to search similar things to help me out. That's why I am asking here.



      So, I have a file:



      path = 'myPath/file.pd0'
      in_file = open(path, "rb")
      read_file = in_file.read()
      type(read_file)


      when I try to check what is inside read_file I get:



      b'x7fx7fxccx05x00x0f$x00`x00xa2x00$x02xe6x02xa8x03xd0x032x04dx04x96x04xa6x04xe0x04'


      The type of read_file is bytes. When I try to use struct since it is the function people suggest I get the following error:



      import struct
      struct.unpack('hhl', read_file[0:30])

      error: unpack requires a buffer of 16 bytes


      No matter what fmt I get unpack requires a buffer of n bytes.



      The file structure that I am trying to read is defined as follow:



      HEADER
      (6 BYTES + [2 x No. OF DATA TYPES])



      FIXED LEADER DATA
      (60 BYTES)



      VARIABLE LEADER DATA
      (66 BYTES)



      CORRELATION MAGNITUDE
      (2 BYTES + 4 BYTES PER DEPTH CELL)



      Any idea how I could start reading these bytes using struct or something similar in python?



      Thank you










      share|improve this question













      I really tried to look for this doubt in many ways but maybe since I never worked with binary files before I don't have any idea what the keywords to search similar things to help me out. That's why I am asking here.



      So, I have a file:



      path = 'myPath/file.pd0'
      in_file = open(path, "rb")
      read_file = in_file.read()
      type(read_file)


      when I try to check what is inside read_file I get:



      b'x7fx7fxccx05x00x0f$x00`x00xa2x00$x02xe6x02xa8x03xd0x032x04dx04x96x04xa6x04xe0x04'


      The type of read_file is bytes. When I try to use struct since it is the function people suggest I get the following error:



      import struct
      struct.unpack('hhl', read_file[0:30])

      error: unpack requires a buffer of 16 bytes


      No matter what fmt I get unpack requires a buffer of n bytes.



      The file structure that I am trying to read is defined as follow:



      HEADER
      (6 BYTES + [2 x No. OF DATA TYPES])



      FIXED LEADER DATA
      (60 BYTES)



      VARIABLE LEADER DATA
      (66 BYTES)



      CORRELATION MAGNITUDE
      (2 BYTES + 4 BYTES PER DEPTH CELL)



      Any idea how I could start reading these bytes using struct or something similar in python?



      Thank you







      python struct binary ascii python-3.6






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 10 at 19:13









      moehbon

      267




      267
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          0
          down vote













          unpack() expects bytes which are the exact length of the format described by its first argument.
          The format string 'hhl' describes data of 16 bytes (on your machine - see below), so you must pass a byte string of 16.
          If you want to parse only part of the bytes, you can do this:



          fmt = 'hhl'
          size = struct.calcsize(fmt)
          struct.unpack(fmt, data[:size])


          Additionally, your format string doesn't have a byte order, size and alignment specifier. It is assumed to be "native" by default. This means your code is system-dependent, which is probably not what you want for parsing a file format.
          You might need different alignments for different parts of the file.






          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53242511%2freading-structured-binary-data-in-python3-6-using-struct%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            0
            down vote













            unpack() expects bytes which are the exact length of the format described by its first argument.
            The format string 'hhl' describes data of 16 bytes (on your machine - see below), so you must pass a byte string of 16.
            If you want to parse only part of the bytes, you can do this:



            fmt = 'hhl'
            size = struct.calcsize(fmt)
            struct.unpack(fmt, data[:size])


            Additionally, your format string doesn't have a byte order, size and alignment specifier. It is assumed to be "native" by default. This means your code is system-dependent, which is probably not what you want for parsing a file format.
            You might need different alignments for different parts of the file.






            share|improve this answer



























              up vote
              0
              down vote













              unpack() expects bytes which are the exact length of the format described by its first argument.
              The format string 'hhl' describes data of 16 bytes (on your machine - see below), so you must pass a byte string of 16.
              If you want to parse only part of the bytes, you can do this:



              fmt = 'hhl'
              size = struct.calcsize(fmt)
              struct.unpack(fmt, data[:size])


              Additionally, your format string doesn't have a byte order, size and alignment specifier. It is assumed to be "native" by default. This means your code is system-dependent, which is probably not what you want for parsing a file format.
              You might need different alignments for different parts of the file.






              share|improve this answer

























                up vote
                0
                down vote










                up vote
                0
                down vote









                unpack() expects bytes which are the exact length of the format described by its first argument.
                The format string 'hhl' describes data of 16 bytes (on your machine - see below), so you must pass a byte string of 16.
                If you want to parse only part of the bytes, you can do this:



                fmt = 'hhl'
                size = struct.calcsize(fmt)
                struct.unpack(fmt, data[:size])


                Additionally, your format string doesn't have a byte order, size and alignment specifier. It is assumed to be "native" by default. This means your code is system-dependent, which is probably not what you want for parsing a file format.
                You might need different alignments for different parts of the file.






                share|improve this answer














                unpack() expects bytes which are the exact length of the format described by its first argument.
                The format string 'hhl' describes data of 16 bytes (on your machine - see below), so you must pass a byte string of 16.
                If you want to parse only part of the bytes, you can do this:



                fmt = 'hhl'
                size = struct.calcsize(fmt)
                struct.unpack(fmt, data[:size])


                Additionally, your format string doesn't have a byte order, size and alignment specifier. It is assumed to be "native" by default. This means your code is system-dependent, which is probably not what you want for parsing a file format.
                You might need different alignments for different parts of the file.







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Nov 10 at 19:42

























                answered Nov 10 at 19:34









                roeen30

                37619




                37619






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53242511%2freading-structured-binary-data-in-python3-6-using-struct%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Full-time equivalent

                    さくらももこ

                    13 indicted, 8 arrested in Calif. drug cartel investigation