How to cause Windbg to download the compressed versions of MS public symbols











up vote
0
down vote

favorite












Microsoft state here:




The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.




I have 2 questions here:




  1. The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
    Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).



  2. and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.



    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
    ...
    ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
    2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]

    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
    ....
    HTTP request sent, awaiting response... 404 Not Found
    2018-11-11 01:17:01 ERROR 404: Not Found.



Update:



I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:




If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.




The question now is how to enable this option in Windbg?



This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.



0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED


However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.










share|improve this question
























  • windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
    – blabb
    Nov 11 at 7:40










  • Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
    – yossizahn
    Nov 11 at 8:43










  • Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
    – Thomas Weller
    Nov 12 at 14:15












  • FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
    – Thomas Weller
    Nov 12 at 14:20












  • @ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
    – yossizahn
    Nov 12 at 20:03















up vote
0
down vote

favorite












Microsoft state here:




The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.




I have 2 questions here:




  1. The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
    Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).



  2. and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.



    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
    ...
    ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
    2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]

    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
    ....
    HTTP request sent, awaiting response... 404 Not Found
    2018-11-11 01:17:01 ERROR 404: Not Found.



Update:



I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:




If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.




The question now is how to enable this option in Windbg?



This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.



0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED


However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.










share|improve this question
























  • windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
    – blabb
    Nov 11 at 7:40










  • Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
    – yossizahn
    Nov 11 at 8:43










  • Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
    – Thomas Weller
    Nov 12 at 14:15












  • FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
    – Thomas Weller
    Nov 12 at 14:20












  • @ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
    – yossizahn
    Nov 12 at 20:03













up vote
0
down vote

favorite









up vote
0
down vote

favorite











Microsoft state here:




The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.




I have 2 questions here:




  1. The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
    Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).



  2. and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.



    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
    ...
    ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
    2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]

    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
    ....
    HTTP request sent, awaiting response... 404 Not Found
    2018-11-11 01:17:01 ERROR 404: Not Found.



Update:



I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:




If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.




The question now is how to enable this option in Windbg?



This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.



0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED


However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.










share|improve this question















Microsoft state here:




The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.




I have 2 questions here:




  1. The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
    Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).



  2. and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.



    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
    ...
    ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
    2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]

    >$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
    ....
    HTTP request sent, awaiting response... 404 Not Found
    2018-11-11 01:17:01 ERROR 404: Not Found.



Update:



I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:




If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.




The question now is how to enable this option in Windbg?



This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.



0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED


However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.







debugging windbg debug-symbols






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 12 at 20:05

























asked Nov 10 at 23:30









yossizahn

13




13












  • windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
    – blabb
    Nov 11 at 7:40










  • Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
    – yossizahn
    Nov 11 at 8:43










  • Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
    – Thomas Weller
    Nov 12 at 14:15












  • FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
    – Thomas Weller
    Nov 12 at 14:20












  • @ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
    – yossizahn
    Nov 12 at 20:03


















  • windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
    – blabb
    Nov 11 at 7:40










  • Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
    – yossizahn
    Nov 11 at 8:43










  • Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
    – Thomas Weller
    Nov 12 at 14:15












  • FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
    – Thomas Weller
    Nov 12 at 14:20












  • @ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
    – yossizahn
    Nov 12 at 20:03
















windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40




windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40












Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43




Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43












Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15






Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15














FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20






FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20














@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03




@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53244453%2fhow-to-cause-windbg-to-download-the-compressed-versions-of-ms-public-symbols%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53244453%2fhow-to-cause-windbg-to-download-the-compressed-versions-of-ms-public-symbols%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Full-time equivalent

さくらももこ

13 indicted, 8 arrested in Calif. drug cartel investigation