why use cell object in python closure implemention?
up vote
2
down vote
favorite
def outer():
n = 1
def inner():
return n
n = 2
return inner
inner = outer()
print innner() # output 2
I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.
python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.
The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.
I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?
python closures cell
asked Nov 11 at 12:21
codinglh
112
add a comment |
up vote
2
down vote
favorite
def outer():
n = 1
def inner():
return n
n = 2
return inner
inner = outer()
print innner() # output 2
I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.
python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.
The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.
I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?
python closures cell
asked Nov 11 at 12:21
codinglh
112
Aaah, it's the same gotcha asfuncs = [(lambda: x) for x in range(3)](known as late binding).
– timgeb
Nov 11 at 12:23
what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
def outer():
n = 1
def inner():
return n
n = 2
return inner
inner = outer()
print innner() # output 2
I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.
python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.
The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.
I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?
python closures cell
asked Nov 11 at 12:21
codinglh
112
def outer():
n = 1
def inner():
return n
n = 2
return inner
inner = outer()
print innner() # output 2
I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.
python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.
The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.
I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?
python closures cell
python closures cell
asked Nov 11 at 12:21
codinglh
112
asked Nov 11 at 12:21
codinglh
112
edited Nov 11 at 14:37
asked Nov 11 at 12:21
codinglh
112
asked Nov 11 at 12:21
codinglh
112
asked Nov 11 at 12:21
codinglh
112
112
Aaah, it's the same gotcha asfuncs = [(lambda: x) for x in range(3)](known as late binding).
– timgeb
Nov 11 at 12:23
what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44
add a comment |
Aaah, it's the same gotcha asfuncs = [(lambda: x) for x in range(3)](known as late binding).
– timgeb
Nov 11 at 12:23
what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44
Aaah, it's the same gotcha as
funcs = [(lambda: x) for x in range(3)] (known as late binding).– timgeb
Nov 11 at 12:23
Aaah, it's the same gotcha as
funcs = [(lambda: x) for x in range(3)] (known as late binding).– timgeb
Nov 11 at 12:23
what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44
what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.
This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.
It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.
answered Nov 12 at 3:08
Davis Herring
7,2871634
Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34
@codinglh: In Python 3 withnonlocalit need not be immutable.
– Davis Herring
Nov 13 at 14:30
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.
This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.
It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.
answered Nov 12 at 3:08
Davis Herring
7,2871634
Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34
@codinglh: In Python 3 withnonlocalit need not be immutable.
– Davis Herring
Nov 13 at 14:30
add a comment |
up vote
0
down vote
The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.
This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.
It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.
answered Nov 12 at 3:08
Davis Herring
7,2871634
Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34
@codinglh: In Python 3 withnonlocalit need not be immutable.
– Davis Herring
Nov 13 at 14:30
add a comment |
up vote
0
down vote
up vote
0
down vote
The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.
This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.
It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.
answered Nov 12 at 3:08
Davis Herring
7,2871634
The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.
This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.
It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.
answered Nov 12 at 3:08
Davis Herring
7,2871634
answered Nov 12 at 3:08
Davis Herring
7,2871634
answered Nov 12 at 3:08
Davis Herring
7,2871634
answered Nov 12 at 3:08
Davis Herring
7,2871634
7,2871634
Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34
@codinglh: In Python 3 withnonlocalit need not be immutable.
– Davis Herring
Nov 13 at 14:30
add a comment |
Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34
@codinglh: In Python 3 withnonlocalit need not be immutable.
– Davis Herring
Nov 13 at 14:30
Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34
Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34
@codinglh: In Python 3 with
nonlocal it need not be immutable.– Davis Herring
Nov 13 at 14:30
@codinglh: In Python 3 with
nonlocal it need not be immutable.– Davis Herring
Nov 13 at 14:30
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53248701%2fwhy-use-cell-object-in-python-closure-implemention%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Aaah, it's the same gotcha as
funcs = [(lambda: x) for x in range(3)](known as late binding).– timgeb
Nov 11 at 12:23
what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44