Discussion:
[Scons-users] TryRun with CacheDir crashes on Python 3
Hector Martin 'marcan'
2018-06-26 11:36:34 UTC
Permalink
Minimal reproducer:

=========================
CacheDir('objects')

env = Environment()

def CompilerCheck(context):
context.Message("Checking for a working C compiler ")
ret = context.TryRun("int a;\n", '.c' )[0]
context.Result(ret)
if ret == 0:
return False

conf = Configure(Environment(),
custom_tests = {"CompilerCheck" : CompilerCheck})

conf.CompilerCheck()
=========================

Output:
scons: Reading SConscript files ...
Checking for a working C compiler TypeError : sequence item 0: expected
str instance, bytes found

I traced it down with a bunch of prints. The crash is in
Node.FS.File.get_cachedir_bsig. sigs ends up looking like this:

[b'int a;\n', '79ff7357d2d39b785bcdf4305604fa0c',
'.sconf_temp/conftest_0.c']

And Util.MD5collect cannot deal with the mixed bytes/str. Not sure what
the right fix is here; either everything has to be str or bytes, or
MD5collect needs to automatically convert when necessary.

SCons by Steven Knight et al.:
script: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668,
2017/11/14 13:16:53, by bdbaddog on hpmicrodog
engine: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668,
2017/11/14 13:16:53, by bdbaddog on hpmicrodog
engine path: ['/usr/lib64/python3.5/site-packages/SCons']
Copyright (c) 2001 - 2017 The SCons Foundation
--
Hector Martin "marcan" (***@marcan.st)
Public Key: https://mrcn.st/pub
Bill Deegan
2018-06-26 15:37:00 UTC
Permalink
Can you rerun with --debug=stacktrace and post.
Post by Hector Martin 'marcan'
=========================
CacheDir('objects')
env = Environment()
context.Message("Checking for a working C compiler ")
ret = context.TryRun("int a;\n", '.c' )[0]
context.Result(ret)
return False
conf = Configure(Environment(),
custom_tests = {"CompilerCheck" : CompilerCheck})
conf.CompilerCheck()
=========================
scons: Reading SConscript files ...
Checking for a working C compiler TypeError : sequence item 0: expected
str instance, bytes found
I traced it down with a bunch of prints. The crash is in
[b'int a;\n', '79ff7357d2d39b785bcdf4305604fa0c',
'.sconf_temp/conftest_0.c']
And Util.MD5collect cannot deal with the mixed bytes/str. Not sure what
the right fix is here; either everything has to be str or bytes, or
MD5collect needs to automatically convert when necessary.
script: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668,
2017/11/14 13:16:53, by bdbaddog on hpmicrodog
engine: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668,
2017/11/14 13:16:53, by bdbaddog on hpmicrodog
engine path: ['/usr/lib64/python3.5/site-packages/SCons']
Copyright (c) 2001 - 2017 The SCons Foundation
--
Public Key: https://mrcn.st/pub
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
Bill Deegan
2018-06-26 15:51:55 UTC
Permalink
Try changing src/engine/SCons/SConf.py line 612 from:

outputStr = self.lastTarget.get_contents().decode()


to:
outputStr = self.lastTarget.get_text_contents()

-Bill
Post by Bill Deegan
Can you rerun with --debug=stacktrace and post.
Post by Hector Martin 'marcan'
=========================
CacheDir('objects')
env = Environment()
context.Message("Checking for a working C compiler ")
ret = context.TryRun("int a;\n", '.c' )[0]
context.Result(ret)
return False
conf = Configure(Environment(),
custom_tests = {"CompilerCheck" : CompilerCheck})
conf.CompilerCheck()
=========================
scons: Reading SConscript files ...
Checking for a working C compiler TypeError : sequence item 0: expected
str instance, bytes found
I traced it down with a bunch of prints. The crash is in
[b'int a;\n', '79ff7357d2d39b785bcdf4305604fa0c',
'.sconf_temp/conftest_0.c']
And Util.MD5collect cannot deal with the mixed bytes/str. Not sure what
the right fix is here; either everything has to be str or bytes, or
MD5collect needs to automatically convert when necessary.
script: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668,
2017/11/14 13:16:53, by bdbaddog on hpmicrodog
engine: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668,
2017/11/14 13:16:53, by bdbaddog on hpmicrodog
engine path: ['/usr/lib64/python3.5/site-packages/SCons']
Copyright (c) 2001 - 2017 The SCons Foundation
--
Public Key: https://mrcn.st/pub
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
Hector Martin 'marcan'
2018-06-26 16:38:02 UTC
Permalink
Post by Bill Deegan
outputStr = self.lastTarget.get_contents().decode()
outputStr = self.lastTarget.get_text_contents()
No change.
Post by Bill Deegan
Can you rerun with --debug=stacktrace and post.
That seems to do nothing. I don't get a stacktrace.
--
Hector Martin "marcan" (***@marcan.st)
Public Key: https://mrcn.st/pub
Bill Deegan
2018-06-26 22:50:39 UTC
Permalink
What's in your config.log file?
Post by Hector Martin 'marcan'
Post by Bill Deegan
outputStr = self.lastTarget.get_contents().decode()
outputStr = self.lastTarget.get_text_contents()
No change.
Post by Bill Deegan
Can you rerun with --debug=stacktrace and post.
That seems to do nothing. I don't get a stacktrace.
--
Public Key: https://mrcn.st/pub
Bill Deegan
2018-06-27 03:47:53 UTC
Permalink
Looks like this is related to a known issue with Value nodes and python 3.
I'm looking into it, but it's a bit tricky because it's in the core.
The good news is the fix will likely fix more than one already filed bug as
well..
Post by Bill Deegan
What's in your config.log file?
Post by Hector Martin 'marcan'
Post by Bill Deegan
outputStr = self.lastTarget.get_contents().decode()
outputStr = self.lastTarget.get_text_contents()
No change.
Post by Bill Deegan
Can you rerun with --debug=stacktrace and post.
That seems to do nothing. I don't get a stacktrace.
--
Public Key: https://mrcn.st/pub
Jason Kenny
2018-06-29 13:57:47 UTC
Permalink
Do you have a fix for this?

Jason

From: Scons-users <scons-users-***@scons.org> On Behalf Of Bill Deegan
Sent: Tuesday, June 26, 2018 10:48 PM
To: Hector Martin 'marcan' <***@marcan.st>
Cc: SCons users mailing list <scons-***@scons.org>
Subject: Re: [Scons-users] TryRun with CacheDir crashes on Python 3

Looks like this is related to a known issue with Value nodes and python 3.
I'm looking into it, but it's a bit tricky because it's in the core.
The good news is the fix will likely fix more than one already filed bug as well..

On Tue, Jun 26, 2018 at 3:50 PM, Bill Deegan <***@baddogconsulting.com<mailto:***@baddogconsulting.com>> wrote:
What's in your config.log file?
Post by Bill Deegan
outputStr = self.lastTarget.get_contents().decode()
outputStr = self.lastTarget.get_text_contents()
No change.
Post by Bill Deegan
Can you rerun with --debug=stacktrace and post.
That seems to do nothing. I don't get a stacktrace.
--
Hector Martin "marcan" (***@marcan.st<mailto:***@marcan.st>)
Public Key: https://mrcn.st/pub<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmrcn.st%2Fpub&data=02%7C01%7C%7Cb12e1fd62a134820843a08d5dbe0c74f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636656680812025262&sdata=jpBTc%2FYfbyJY2vpXu%2FG2C5lATnaSOlPRc%2BaUstbu0j4%3D&reserved=0>
Bill Deegan
2018-06-29 16:09:12 UTC
Permalink
Working on it.
Got distracted by paying work...;)
Post by Jason Kenny
Do you have a fix for this?
Jason
Deegan
*Sent:* Tuesday, June 26, 2018 10:48 PM
*Subject:* Re: [Scons-users] TryRun with CacheDir crashes on Python 3
Looks like this is related to a known issue with Value nodes and python 3.
I'm looking into it, but it's a bit tricky because it's in the core.
The good news is the fix will likely fix more than one already filed bug as well..
What's in your config.log file?
Post by Bill Deegan
outputStr = self.lastTarget.get_contents().decode()
outputStr = self.lastTarget.get_text_contents()
No change.
Post by Bill Deegan
Can you rerun with --debug=stacktrace and post.
That seems to do nothing. I don't get a stacktrace.
--
Public Key: https://mrcn.st/pub
<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmrcn.st%2Fpub&data=02%7C01%7C%7Cb12e1fd62a134820843a08d5dbe0c74f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636656680812025262&sdata=jpBTc%2FYfbyJY2vpXu%2FG2C5lATnaSOlPRc%2BaUstbu0j4%3D&reserved=0>
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
Hector Martin 'marcan'
2018-06-27 03:55:22 UTC
Permalink
Post by Bill Deegan
What's in your config.log file?
Nothing useful, just the one test

$ cat config.log
file /home/marcan/tmp/sc/SConstruct,line 13:
Configure(confdir = .sconf_temp)
scons: Configure: Checking for a working C compiler
.sconf_temp/conftest_0.c <-
|int a;
|

TBH, I think that's a bug in and of itself - there seems to be no way of
debugging this problem or getting a stacktrace other than hacking up the
SCons code with prints like I did.
--
Hector Martin "marcan" (***@marcan.st)
Public Key: https://mrcn.st/pub
Bill Deegan
2018-06-27 04:03:22 UTC
Permalink
The issue is that the exception is getting swallowed for later output which
is normally what you want, but in this case it's not.
Anyway it's really a python 3 port issue which was no resolved.
So I wouldn't expect it to be super easy to debug at this point.

The TryRun checker creates a Value() node internally.
In creating a signature for the Value() node to determine if there's an
object in the cache an error is exercised.
Post by Hector Martin 'marcan'
Post by Bill Deegan
What's in your config.log file?
Nothing useful, just the one test
$ cat config.log
Configure(confdir = .sconf_temp)
scons: Configure: Checking for a working C compiler
.sconf_temp/conftest_0.c <-
|int a;
|
TBH, I think that's a bug in and of itself - there seems to be no way of
debugging this problem or getting a stacktrace other than hacking up the
SCons code with prints like I did.
--
Public Key: https://mrcn.st/pub
Loading...