How can I execute Lua functions from C++ without having to find them each time?
up vote
1
down vote
favorite
I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.
Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.
Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.
Thanks in advance
lua evaluation
|
show 7 more comments
up vote
1
down vote
favorite
I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.
Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.
Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.
Thanks in advance
lua evaluation
"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42
Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09
Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53
Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56
"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you doadd(...)
in Lua script, the Lua interpreter is going to do the exact same work as your C code (unlessadd
is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whateverlua_ref
gimmicks you use won't help.
– Nicol Bolas
Jun 5 '16 at 13:29
|
show 7 more comments
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.
Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.
Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.
Thanks in advance
lua evaluation
I'm new to Lua, figuring out how to embed/use it from a C++ program. I have the basic mechanism working but interested in code being executed as fast as possible. So I am loading/compiling lua code (containing multiple functions) that I can use later.
Now suppose I have a lua function called 'add' that I want to call from C++. As far as I can tell, I have to use the lua_getglobal() function which seems to find and push the bytecode for 'add' on to the Lua stack. Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it? I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization but I am trying to build a hard real-time system (yes, actual physical deadlines, not just fast) and if I have to call 'add' hundreds of thousands of times in a hurry, having to look for the function every time seems like a real waste.
Also, for curiosity, does the actual bytecode get pushed (i.e, copied) on to the stack or just a reference to it? I'm hoping the latter.
Thanks in advance
lua evaluation
lua evaluation
asked Jun 5 '16 at 2:41
David
2,50432334
2,50432334
"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42
Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09
Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53
Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56
"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you doadd(...)
in Lua script, the Lua interpreter is going to do the exact same work as your C code (unlessadd
is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whateverlua_ref
gimmicks you use won't help.
– Nicol Bolas
Jun 5 '16 at 13:29
|
show 7 more comments
"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42
Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09
Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53
Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56
"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you doadd(...)
in Lua script, the Lua interpreter is going to do the exact same work as your C code (unlessadd
is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whateverlua_ref
gimmicks you use won't help.
– Nicol Bolas
Jun 5 '16 at 13:29
"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42
"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42
Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09
Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09
Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53
Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53
Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56
Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56
"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do
add(...)
in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add
is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref
gimmicks you use won't help.– Nicol Bolas
Jun 5 '16 at 13:29
"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do
add(...)
in Lua script, the Lua interpreter is going to do the exact same work as your C code (unless add
is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whatever lua_ref
gimmicks you use won't help.– Nicol Bolas
Jun 5 '16 at 13:29
|
show 7 more comments
2 Answers
2
active
oldest
votes
up vote
2
down vote
accepted
lua_getglobal
, at its core, is just doing a table lookup, functionally no different from doing lua_getfield
. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.
That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.
As for being able to store the function somewhere for faster access... no. Well, not reasonably.
Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer
, but there's no way to undo that conversion.
However, what you could do is create a subsidiary lua_State
from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove
to move the function from the subsidiary lua_State
into your main state to be executed.
Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add")
is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs)
calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.
Also, it's not clear how big a lua_State
's stack is allowed to be. So you may not have enough space to make that work.
Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.
Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12
1
@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47
Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06
@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25
Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03
add a comment |
up vote
4
down vote
Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?
One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.
In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0
http://lua-users.org/wiki/OptimisationCodingTips
I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1
http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref
Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04
Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
lua_getglobal
, at its core, is just doing a table lookup, functionally no different from doing lua_getfield
. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.
That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.
As for being able to store the function somewhere for faster access... no. Well, not reasonably.
Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer
, but there's no way to undo that conversion.
However, what you could do is create a subsidiary lua_State
from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove
to move the function from the subsidiary lua_State
into your main state to be executed.
Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add")
is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs)
calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.
Also, it's not clear how big a lua_State
's stack is allowed to be. So you may not have enough space to make that work.
Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.
Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12
1
@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47
Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06
@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25
Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03
add a comment |
up vote
2
down vote
accepted
lua_getglobal
, at its core, is just doing a table lookup, functionally no different from doing lua_getfield
. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.
That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.
As for being able to store the function somewhere for faster access... no. Well, not reasonably.
Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer
, but there's no way to undo that conversion.
However, what you could do is create a subsidiary lua_State
from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove
to move the function from the subsidiary lua_State
into your main state to be executed.
Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add")
is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs)
calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.
Also, it's not clear how big a lua_State
's stack is allowed to be. So you may not have enough space to make that work.
Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.
Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12
1
@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47
Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06
@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25
Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03
add a comment |
up vote
2
down vote
accepted
up vote
2
down vote
accepted
lua_getglobal
, at its core, is just doing a table lookup, functionally no different from doing lua_getfield
. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.
That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.
As for being able to store the function somewhere for faster access... no. Well, not reasonably.
Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer
, but there's no way to undo that conversion.
However, what you could do is create a subsidiary lua_State
from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove
to move the function from the subsidiary lua_State
into your main state to be executed.
Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add")
is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs)
calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.
Also, it's not clear how big a lua_State
's stack is allowed to be. So you may not have enough space to make that work.
Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.
lua_getglobal
, at its core, is just doing a table lookup, functionally no different from doing lua_getfield
. Lua tables are hashtables, so lookup is amortized constant time regardless of the number of entries.
That doesn't mean that such fetches are super-fast. But the performance of fetching more-or-less does not depend on how many different items are in the table.
As for being able to store the function somewhere for faster access... no. Well, not reasonably.
Lua values cannot be stored in C code. While you can get numbers or strings from Lua values, you cannot get a Lua function in any meaningful way. You could call lua_topointer
, but there's no way to undo that conversion.
However, what you could do is create a subsidiary lua_State
from the main one. You would then load up its stack with functions. Using a stack index is faster than doing a hash-table lookup. When it comes time to call this function, you would use lua_xmove
to move the function from the subsidiary lua_State
into your main state to be executed.
Of course, doing this makes your code more difficult to read. A simple lua_getglobal(L, "add")
is much more meaningful than lua_pushvalue(Funcs, some_int)/lua_xmove(L, Funcs)
calls. Using the global table also permits you to change that value, causing all accesses to it to produce a new function.
Also, it's not clear how big a lua_State
's stack is allowed to be. So you may not have enough space to make that work.
Lastly, accessing a function does not copy actual memory. But putting a garbage collected object on the stack means that its GC data has to be updated to note that it is being referenced by the stack. So it's not just a pointer copy.
edited Jun 5 '16 at 14:46
answered Jun 5 '16 at 4:09
Nicol Bolas
279k33457629
279k33457629
Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12
1
@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47
Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06
@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25
Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03
add a comment |
Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12
1
@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47
Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06
@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25
Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03
Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12
Nicol, thanks so much for your very useful feedback. Readability issue understood, but such things can be carefully commented and kept isolated in some C++ inlined methods to manage them. However, reading up on lua_xmove, it's unclear to me how this helps. Documentation suggests that this actually pops from one lua_State to put on the other. Would that not remove the function from the subsidiary stack?
– David
Jun 5 '16 at 11:12
1
1
@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47
@David The point is probably to have the value in the other stack, then make a copy of it there with pushvalue and then move that copy. That way you dont completely remove it from the second stack. My small test code however shows that the difference with using 2 stacks and xmove VS the luaL_ref would only be about 1.2 times speed up. Updated my test code to show this also.
– Rochet2
Jun 5 '16 at 11:47
Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06
Got it. Please understand that since I'm only starting to look at Lua (among other things), I'm not deeply familiar with the appropriate idioms. The notion of a stack in which items can be accessed/removed when they're not at the top was something of which I was unaware (nor would I consider such a structure to be called a stack, for that matter)
– David
Jun 5 '16 at 15:06
@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25
@David: "nor would I consider such a structure to be called a stack, for that matter" That's an interesting element of Lua. The Lua stack doubles as Lua's register file. So if a Lua script needs to access some local variable that was created earlier, then it can reach down into the "stack" and retrieve it, no differently than if it were at the top. It just uses a different index. Its much more convenient than you might think.
– Nicol Bolas
Jun 5 '16 at 15:25
Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03
Yes, I have no doubt it is very convenient ---- but it shouldn't be called a stack, because technically, it isn't, and that's why your suggestion to use lua_xmove was confusing to me. By definition, a stack has push, pop, isempty operations, maybe top for peeking. If there's anything else, it ain't a stack, even if it includes stack functionality.
– David
Jun 5 '16 at 18:03
add a comment |
up vote
4
down vote
Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?
One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.
In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0
http://lua-users.org/wiki/OptimisationCodingTips
I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1
http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref
Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04
Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06
add a comment |
up vote
4
down vote
Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?
One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.
In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0
http://lua-users.org/wiki/OptimisationCodingTips
I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1
http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref
Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04
Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06
add a comment |
up vote
4
down vote
up vote
4
down vote
Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?
One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.
In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0
http://lua-users.org/wiki/OptimisationCodingTips
I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1
http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref
Can I eliminate the finding part, i.e. can I keep a reference to the bytecode for the function 'add' so that I can simply push it on to the stack when I need to use it?
One thing you can try is to use luaL_Ref to create a reference to the function.
This reference should be faster to access - seems it uses a number indexed table so it could possibly be faster than a hash table with string keys.
In C, use lua_ref() wherever possible. lua_ref() behaves similarly to a local variable in terms of speed. - Lua, pre v4.0
http://lua-users.org/wiki/OptimisationCodingTips
I tried to do some tests and with 1000000000 iterations of tonumber pushed to stack and the results were that lua_getglobal took 24 seconds, rawgeti with the ref took 12 seconds, 2 stacks and lua_pushvalue + lua_xmove took 11 seconds and using lua_pushvalue with the same state and the actual function being at stack top took 6 seconds.
See code here: http://pastebin.com/n8Q0uVx1
http://www.lua.org/manual/5.3/manual.html#luaL_ref
http://www.lua.org/manual/5.3/manual.html#luaL_unref
edited Jun 5 '16 at 11:49
answered Jun 5 '16 at 10:38
Rochet2
7871511
7871511
Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04
Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06
add a comment |
Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04
Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06
Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04
Very interesting. Unless I missed something, one thing that concerns me about the Lua implementation is that compiled functions (bytechunks, which presumably don't change?) have to be copied to the stack every time, rather than just pushing the address (say) of that code. If your function is large, doing that copy many times seems very inefficient.
– David
Jun 5 '16 at 11:04
Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06
Ah, just read Nicol Bolas' second comment where he notes at the bottom (if I understand correctly) that the bytechunk doesn't get copied.
– David
Jun 5 '16 at 11:06
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f37637935%2fhow-can-i-execute-lua-functions-from-c-without-having-to-find-them-each-time%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
"I don't know how efficient lua_getglobal() is when there are thousands of lua functions, nor am I a fan of premature optimization" Premature optimization is any optimization made without knowing the performance of the original operation to be optimized or of the program overall. So you seem to be contradicting yourself.
– Nicol Bolas
Jun 5 '16 at 3:42
Also, if you're building a "hard real-time system", then you should stay away from any language with garbage collection. Like, you know, Lua.
– Nicol Bolas
Jun 5 '16 at 4:09
Appreciate the responses. Re: premature optimization. Yes, I know what it is. But it's also the case that anyone with experience is aware of legitimate choke-points, such as the inside of a nested 'for' loop that can get executed millions of times. Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated.
– David
Jun 5 '16 at 10:53
Re: GC, yeah, I'm aware of the GC concern. There are some deterministic GCs out there but I don't yet know enough about Lua to know whether it will be good enough. I only started looking at embedded scripting languages a couple of days ago and have several others to check out, such as ChaiScript, which I haven't examined at all yet.
– David
Jun 5 '16 at 10:56
"Wasn't planning to instantly get rid of the 'find' part but it's useful to know whether it CAN be eliminated." The thing you're not understanding is that every time you do
add(...)
in Lua script, the Lua interpreter is going to do the exact same work as your C code (unlessadd
is a local variable). So unless the only Lua functions you call are functions that don't do anything, the performance of fetching a function in C is irrelevant. This is another reason why premature optimization is wrong: you're looking in the wrong place. So whateverlua_ref
gimmicks you use won't help.– Nicol Bolas
Jun 5 '16 at 13:29