How can I execute Lua functions from C++ without having to find them each time?











up vote
1
down vote

favorite












I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.



Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.



Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.



Thanks in advance










share|improve this question






















  • "I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
    – Nicol Bolas
    Jun 5 '16 at 3:42










  • Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
    – Nicol Bolas
    Jun 5 '16 at 4:09










  • Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
    – David
    Jun 5 '16 at 10:53










  • Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
    – David
    Jun 5 '16 at 10:56










  • "Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do add(...) in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref gimmicks you use won't help.
    – Nicol Bolas
    Jun 5 '16 at 13:29















up vote
1
down vote

favorite












I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.



Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.



Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.



Thanks in advance










share|improve this question






















  • "I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
    – Nicol Bolas
    Jun 5 '16 at 3:42










  • Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
    – Nicol Bolas
    Jun 5 '16 at 4:09










  • Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
    – David
    Jun 5 '16 at 10:53










  • Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
    – David
    Jun 5 '16 at 10:56










  • "Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do add(...) in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref gimmicks you use won't help.
    – Nicol Bolas
    Jun 5 '16 at 13:29













up vote
1
down vote

favorite









up vote
1
down vote

favorite











I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.



Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.



Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.



Thanks in advance










share|improve this question













I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.



Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.



Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.



Thanks in advance







lua evaluation






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Jun 5 '16 at 2:41









David

2,50432334




2,50432334












  • "I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
    – Nicol Bolas
    Jun 5 '16 at 3:42










  • Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
    – Nicol Bolas
    Jun 5 '16 at 4:09










  • Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
    – David
    Jun 5 '16 at 10:53










  • Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
    – David
    Jun 5 '16 at 10:56










  • "Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do add(...) in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref gimmicks you use won't help.
    – Nicol Bolas
    Jun 5 '16 at 13:29


















  • "I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
    – Nicol Bolas
    Jun 5 '16 at 3:42










  • Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
    – Nicol Bolas
    Jun 5 '16 at 4:09










  • Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
    – David
    Jun 5 '16 at 10:53










  • Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
    – David
    Jun 5 '16 at 10:56










  • "Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do add(...) in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref gimmicks you use won't help.
    – Nicol Bolas
    Jun 5 '16 at 13:29
















"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42




"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42












Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09




Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09












Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53




Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53












Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56




Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56












"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do add(...) in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref gimmicks you use won't help.
– Nicol Bolas
Jun 5 '16 at 13:29




"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do add(...) in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref gimmicks you use won't help.
– Nicol Bolas
Jun 5 '16 at 13:29












2 Answers
2






active

oldest

votes

















up vote
2
down vote



accepted










lua_getglobal, at its core, is just doing a table lookup, functionally no different from doing lua_getfield. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.



That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.



As for being able to store the function somewhere for faster access... no. Well, not reasonably.



Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer, but there's no way to undo that conversion.



However, what you could do is create a subsidiary lua_State from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove to move the function from the subsidiary lua_State into your main state to be executed.



Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add") is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs) calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.



Also, it's not clear how big a lua_State's stack is allowed to be. So you may not have enough space to make that work.



Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.






share|improve this answer























  • Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
    – David
    Jun 5 '16 at 11:12






  • 1




    @David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
    – Rochet2
    Jun 5 '16 at 11:47












  • Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
    – David
    Jun 5 '16 at 15:06










  • @David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
    – Nicol Bolas
    Jun 5 '16 at 15:25










  • Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
    – David
    Jun 5 '16 at 18:03


















up vote
4
down vote














Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?




One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.




In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0



http://lua-users.org/wiki/OptimisationCodingTips




I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1



http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref






share|improve this answer























  • Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
    – David
    Jun 5 '16 at 11:04










  • Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
    – David
    Jun 5 '16 at 11:06











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f37637935%2fhow-can-i-execute-lua-functions-from-c-without-having-to-find-them-each-time%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
2
down vote



accepted










lua_getglobal, at its core, is just doing a table lookup, functionally no different from doing lua_getfield. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.



That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.



As for being able to store the function somewhere for faster access... no. Well, not reasonably.



Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer, but there's no way to undo that conversion.



However, what you could do is create a subsidiary lua_State from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove to move the function from the subsidiary lua_State into your main state to be executed.



Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add") is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs) calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.



Also, it's not clear how big a lua_State's stack is allowed to be. So you may not have enough space to make that work.



Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.






share|improve this answer























  • Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
    – David
    Jun 5 '16 at 11:12






  • 1




    @David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
    – Rochet2
    Jun 5 '16 at 11:47












  • Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
    – David
    Jun 5 '16 at 15:06










  • @David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
    – Nicol Bolas
    Jun 5 '16 at 15:25










  • Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
    – David
    Jun 5 '16 at 18:03















up vote
2
down vote



accepted










lua_getglobal, at its core, is just doing a table lookup, functionally no different from doing lua_getfield. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.



That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.



As for being able to store the function somewhere for faster access... no. Well, not reasonably.



Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer, but there's no way to undo that conversion.



However, what you could do is create a subsidiary lua_State from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove to move the function from the subsidiary lua_State into your main state to be executed.



Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add") is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs) calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.



Also, it's not clear how big a lua_State's stack is allowed to be. So you may not have enough space to make that work.



Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.






share|improve this answer























  • Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
    – David
    Jun 5 '16 at 11:12






  • 1




    @David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
    – Rochet2
    Jun 5 '16 at 11:47












  • Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
    – David
    Jun 5 '16 at 15:06










  • @David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
    – Nicol Bolas
    Jun 5 '16 at 15:25










  • Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
    – David
    Jun 5 '16 at 18:03













up vote
2
down vote



accepted







up vote
2
down vote



accepted






lua_getglobal, at its core, is just doing a table lookup, functionally no different from doing lua_getfield. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.



That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.



As for being able to store the function somewhere for faster access... no. Well, not reasonably.



Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer, but there's no way to undo that conversion.



However, what you could do is create a subsidiary lua_State from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove to move the function from the subsidiary lua_State into your main state to be executed.



Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add") is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs) calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.



Also, it's not clear how big a lua_State's stack is allowed to be. So you may not have enough space to make that work.



Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.






share|improve this answer














lua_getglobal, at its core, is just doing a table lookup, functionally no different from doing lua_getfield. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.



That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.



As for being able to store the function somewhere for faster access... no. Well, not reasonably.



Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer, but there's no way to undo that conversion.



However, what you could do is create a subsidiary lua_State from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove to move the function from the subsidiary lua_State into your main state to be executed.



Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add") is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs) calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.



Also, it's not clear how big a lua_State's stack is allowed to be. So you may not have enough space to make that work.



Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.







share|improve this answer














share|improve this answer



share|improve this answer








edited Jun 5 '16 at 14:46

























answered Jun 5 '16 at 4:09









Nicol Bolas

279k33457629




279k33457629












  • Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
    – David
    Jun 5 '16 at 11:12






  • 1




    @David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
    – Rochet2
    Jun 5 '16 at 11:47












  • Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
    – David
    Jun 5 '16 at 15:06










  • @David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
    – Nicol Bolas
    Jun 5 '16 at 15:25










  • Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
    – David
    Jun 5 '16 at 18:03


















  • Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
    – David
    Jun 5 '16 at 11:12






  • 1




    @David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
    – Rochet2
    Jun 5 '16 at 11:47












  • Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
    – David
    Jun 5 '16 at 15:06










  • @David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
    – Nicol Bolas
    Jun 5 '16 at 15:25










  • Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
    – David
    Jun 5 '16 at 18:03
















Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12




Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12




1




1




@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47






@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47














Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06




Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06












@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25




@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25












Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03




Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03












up vote
4
down vote














Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?




One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.




In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0



http://lua-users.org/wiki/OptimisationCodingTips




I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1



http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref






share|improve this answer























  • Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
    – David
    Jun 5 '16 at 11:04










  • Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
    – David
    Jun 5 '16 at 11:06















up vote
4
down vote














Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?




One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.




In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0



http://lua-users.org/wiki/OptimisationCodingTips




I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1



http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref






share|improve this answer























  • Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
    – David
    Jun 5 '16 at 11:04










  • Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
    – David
    Jun 5 '16 at 11:06













up vote
4
down vote










up vote
4
down vote










Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?




One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.




In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0



http://lua-users.org/wiki/OptimisationCodingTips




I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1



http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref






share|improve this answer















Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?




One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.




In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0



http://lua-users.org/wiki/OptimisationCodingTips




I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1



http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref







share|improve this answer














share|improve this answer



share|improve this answer








edited Jun 5 '16 at 11:49

























answered Jun 5 '16 at 10:38









Rochet2

7871511




7871511












  • Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
    – David
    Jun 5 '16 at 11:04










  • Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
    – David
    Jun 5 '16 at 11:06


















  • Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
    – David
    Jun 5 '16 at 11:04










  • Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
    – David
    Jun 5 '16 at 11:06
















Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04




Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04












Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06




Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06


















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f37637935%2fhow-can-i-execute-lua-functions-from-c-without-having-to-find-them-each-time%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Full-time equivalent

Bicuculline

さくらももこ