[Scons-users] using variant_dir but strip out the variant

I finally implemented something like _concat() so I can use it
directly in the COMSTR definition to strip out the additional path.
However now scons trying to be very verbose and every single
os.chdir() is being printed (chdir="..." used for all builders). Is
there a way to disable this os.chdir() print message without modifying
the scons source code? Thanks.

Post by Hua Yanghao
Dear Scons User,
I am having maybe an odd request. The background is there are some
compilers out there produces all kinds of output in your current
directory which you do not have control. I wanted to combine the
variant_dir feature plus the chdir kw arguments to builders, to be
able to invoke those compilers directly inside the variant_dir. And I
still want to keep the source duplication in the variant_dir as it
will be a snapshot of only the used files for a particular built out
of thousands of irrelevant files. As a side effect now the source
files are all prefixed with the variant_dir which becomes "invalid"
because the compiler is already invoked in the variant_dir instead of
the top level directory.
Is there a way (e.g. a special env variable) already that when I write
my builders instead of "$COMPILER $SOURCE", I can use something like
"$COMPILER $ORIGINAL_SOURCE"? Or do I have to wrap around the
$COMPILER with another command such that I can do things like
"$COMPILER_WRAPPED $VARIANT_DIR $SOURCE"? I know the second one
definitely can work but wondering if there are better ways already in
scons.
Best Regards,
Yanghao

Hua Yanghao

2018-10-17 22:07:16 UTC

Looked into the Action.py a little bit, doesn't seem to really have a
controllable print for chdir messages.
I tested below patch which works for me:

diff --git i/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
w/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
index ce5471d..ee2a5c1 100644
--- i/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
+++ w/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
@@ -647,7 +647,8 @@ class _ActionAction(ActionBase):
show=_null,
execute=_null,
chdir=_null,
- executor=None):
+ executor=None,
+ chdir_verbose=False):
if not is_List(target):
target = [target]
if not is_List(source):
@@ -691,7 +692,8 @@ class _ActionAction(ActionBase):
cmd = self.strfunction(target, source, env)
if cmd:
if chdir:
- cmd = ('os.chdir(%s)\n' % repr(chdir)) + cmd
+ if chdir_verbose:
+ cmd = ('os.chdir(%s)\n' % repr(chdir)) + cmd
try:
get = env.get
except AttributeError:
@@ -718,7 +720,7 @@ class _ActionAction(ActionBase):
finally:
if save_cwd:
os.chdir(save_cwd)
- if cmd and save_cwd:
+ if cmd and save_cwd and chdir_verbose:
print_func('os.chdir(%s)' % repr(save_cwd), target, source, env)

return stat

Post by Hua Yanghao
I finally implemented something like _concat() so I can use it
directly in the COMSTR definition to strip out the additional path.
However now scons trying to be very verbose and every single
os.chdir() is being printed (chdir="..." used for all builders). Is
there a way to disable this os.chdir() print message without modifying
the scons source code? Thanks.

Hua Yanghao

2018-10-18 13:30:28 UTC

Update: it looks like the _concat()-like stuff can only be put in the
SCons/Defaults.py otherwise not all environment will get it.
Not sure if there is a way that I do not need to modify the
Defaults.py but still able to add one function to the default
environment construction.

Thanks,
Yanghao

Post by Hua Yanghao
Looked into the Action.py a little bit, doesn't seem to really have a
controllable print for chdir messages.
diff --git i/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
w/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
index ce5471d..ee2a5c1 100644
--- i/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
+++ w/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
show=_null,
execute=_null,
chdir=_null,
+ executor=None,
target = [target]
cmd = self.strfunction(target, source, env)
- cmd = ('os.chdir(%s)\n' % repr(chdir)) + cmd
+ cmd = ('os.chdir(%s)\n' % repr(chdir)) + cmd
get = env.get
os.chdir(save_cwd)
print_func('os.chdir(%s)' % repr(save_cwd), target, source, env)
return stat

Bill Deegan

2018-10-18 16:04:19 UTC

So you have a file:
a/b/c.c
variant dir to
build/a/b/c.c

you're chdir'ing into
build/a/b ?

And you want the command line to be
compiler c.c

?

Post by Hua Yanghao
Update: it looks like the _concat()-like stuff can only be put in the
SCons/Defaults.py otherwise not all environment will get it.
Not sure if there is a way that I do not need to modify the
Defaults.py but still able to add one function to the default
environment construction.
Thanks,
Yanghao

env)

Post by Hua Yanghao
return stat

write

Post by Hua Yanghao
my builders instead of "$COMPILER $SOURCE", I can use something like
"$COMPILER $ORIGINAL_SOURCE"? Or do I have to wrap around the
$COMPILER with another command such that I can do things like
"$COMPILER_WRAPPED $VARIANT_DIR $SOURCE"? I know the second one
definitely can work but wondering if there are better ways already in
scons.
Best Regards,
Yanghao

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Hua Yanghao

2018-10-18 16:06:46 UTC

Hi Bill, I am chdir only to the top level folder "build", and want to
build CC a/b/c.c instead of CC build/a/b/c.c

I got this working but wonder if there is a way that I do not need to
change the Defaults.py file.

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Hua Yanghao

2018-10-24 15:04:16 UTC

Any thoughts? This seems really like when you move around some header
files scons is using some cached knowledge from last built which
caused the incremental built to fail.
Not sure if the trace file is useful enough already, I could try to
create a minimal suite of files that actually reproduce this issue too
if needed.

Post by Hua Yanghao
Hi Bill, I am chdir only to the top level folder "build", and want to
build CC a/b/c.c instead of CC build/a/b/c.c
I got this working but wonder if there is a way that I do not need to
change the Defaults.py file.

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Bill Deegan

2018-10-24 18:14:25 UTC

you should not have to change defaults.py (ever.. it's python you can reach
into anything and change anything in your SConstruct/sconscript/etc)

So you want the string to be a/b/c.c and not build/a/b/c.c

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

source, env)

Post by Hua Yanghao
return stat

modifying

Post by Hua Yanghao
the scons source code? Thanks.

And I

Post by Hua Yanghao
still want to keep the source duplication in the variant_dir as it
will be a snapshot of only the used files for a particular built

out

Post by Hua Yanghao
of thousands of irrelevant files. As a side effect now the source
files are all prefixed with the variant_dir which becomes

"invalid"

Post by Hua Yanghao
because the compiler is already invoked in the variant_dir

instead of

Post by Hua Yanghao
the top level directory.
Is there a way (e.g. a special env variable) already that when I

write

Post by Hua Yanghao
my builders instead of "$COMPILER $SOURCE", I can use something

Post by Hua Yanghao
"$COMPILER $ORIGINAL_SOURCE"? Or do I have to wrap around the
$COMPILER with another command such that I can do things like
"$COMPILER_WRAPPED $VARIANT_DIR $SOURCE"? I know the second one
definitely can work but wondering if there are better ways

already in

Post by Hua Yanghao
scons.
Best Regards,
Yanghao

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Hua Yanghao

2018-10-24 20:02:39 UTC

Bill, I tried but enlighten me please.

The problem is when scons took the defaults.py settings ... is that
before the first line of code in SConstruct is being called or after
... my test shows it is the latter so whatever I change in
SConstruct/SConscript does not make the function exported to the
environment where the command string is expanded ...

you should not have to change defaults.py (ever.. it's python you can reach into anything and change anything in your SConstruct/sconscript/etc)
So you want the string to be a/b/c.c and not build/a/b/c.c

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Hua Yanghao

2018-10-24 20:03:41 UTC

correction: my test shows it is the *former*.

Post by Hua Yanghao
Bill, I tried but enlighten me please.
The problem is when scons took the defaults.py settings ... is that
before the first line of code in SConstruct is being called or after
... my test shows it is the latter so whatever I change in
SConstruct/SConscript does not make the function exported to the
environment where the command string is expanded ...

you should not have to change defaults.py (ever.. it's python you can reach into anything and change anything in your SConstruct/sconscript/etc)
So you want the string to be a/b/c.c and not build/a/b/c.c

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Bill Deegan

2018-10-24 23:45:26 UTC

Did you try to monkey path it in site_scons/site_init.py ?

Do you create new Environment()'s all over the place?
Or do you create a few and then clone them places?
You can set env['_concat']

Hacking default.py is a bad idea in general, and a very bad idea unless
your using the scons-local scons package included with your sources (in
which case it's still a bad idea)

Post by Hua Yanghao
correction: my test shows it is the *former*.

Post by Bill Deegan
you should not have to change defaults.py (ever.. it's python you can

reach into anything and change anything in your SConstruct/sconscript/etc)

Post by Bill Deegan
So you want the string to be a/b/c.c and not build/a/b/c.c

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

Post by Hua Yanghao
Update: it looks like the _concat()-like stuff can only be put in

the

Post by Hua Yanghao
SCons/Defaults.py otherwise not all environment will get it.
Not sure if there is a way that I do not need to modify the
Defaults.py but still able to add one function to the default
environment construction.
Thanks,
Yanghao

Post by Hua Yanghao
Looked into the Action.py a little bit, doesn't seem to really

have a

Post by Hua Yanghao
controllable print for chdir messages.
diff --git i/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
w/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
index ce5471d..ee2a5c1 100644
--- i/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
+++ w/tools/scons-3.0.1/scons-local-3.0.1/SCons/Action.py
show=_null,
execute=_null,
chdir=_null,
+ executor=None,
target = [target]
cmd = self.strfunction(target, source, env)
- cmd = ('os.chdir(%s)\n' % repr(chdir)) + cmd
+ cmd = ('os.chdir(%s)\n' % repr(chdir))

+ cmd

Post by Hua Yanghao
get = env.get
os.chdir(save_cwd)
print_func('os.chdir(%s)' % repr(save_cwd), target,

source, env)

Post by Hua Yanghao
return stat
On Wed, Oct 17, 2018 at 11:54 PM Hua Yanghao <

Post by Hua Yanghao
I finally implemented something like _concat() so I can use it
directly in the COMSTR definition to strip out the additional

path.

Post by Hua Yanghao
However now scons trying to be very verbose and every single
os.chdir() is being printed (chdir="..." used for all

builders). Is

Post by Hua Yanghao
there a way to disable this os.chdir() print message without

modifying

Post by Hua Yanghao
the scons source code? Thanks.
On Wed, Oct 17, 2018 at 9:55 PM Hua Yanghao <

Post by Hua Yanghao
Dear Scons User,
I am having maybe an odd request. The background is there

are some

Post by Hua Yanghao
compilers out there produces all kinds of output in your

current

Post by Hua Yanghao
directory which you do not have control. I wanted to combine

the

Post by Hua Yanghao
variant_dir feature plus the chdir kw arguments to builders,

to be

Post by Hua Yanghao
able to invoke those compilers directly inside the

variant_dir. And I

Post by Hua Yanghao
still want to keep the source duplication in the variant_dir

as it

Post by Hua Yanghao
will be a snapshot of only the used files for a particular

built out

Post by Hua Yanghao
of thousands of irrelevant files. As a side effect now the

source

Post by Hua Yanghao
files are all prefixed with the variant_dir which becomes

"invalid"

Post by Hua Yanghao
because the compiler is already invoked in the variant_dir

instead of

Post by Hua Yanghao
the top level directory.
Is there a way (e.g. a special env variable) already that

when I write

Post by Hua Yanghao
my builders instead of "$COMPILER $SOURCE", I can use

something like

one

Post by Hua Yanghao
definitely can work but wondering if there are better ways

already in

Post by Hua Yanghao
scons.
Best Regards,
Yanghao

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Hua Yanghao

2018-10-25 08:19:47 UTC

Hi Bill,
I know it is a bad bad idea that's why I am asking for a better way :)

I almost forgot we have site_scons/site_init.py to play with. I create
quite a few different env but I guess if site_init.py is loaded fast
enough I should be able to override it there ...
Will let you know the result.

BR, Yanghao

Post by Bill Deegan
Did you try to monkey path it in site_scons/site_init.py ?
Do you create new Environment()'s all over the place?
Or do you create a few and then clone them places?
You can set env['_concat']
Hacking default.py is a bad idea in general, and a very bad idea unless your using the scons-local scons package included with your sources (in which case it's still a bad idea)

Post by Hua Yanghao
correction: my test shows it is the *former*.

you should not have to change defaults.py (ever.. it's python you can reach into anything and change anything in your SConstruct/sconscript/etc)
So you want the string to be a/b/c.c and not build/a/b/c.c

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Hua Yanghao

2018-10-25 08:58:34 UTC

site_init.py worked fine and modification are effective for all
environments later created. Thanks Bill.

1 import os
2 from SCons import Defaults
3
4 def _usw_relpath(sources, base, env):
5 #print("#"*30)
6 #print(str(sources))
7 p = str(sources)
8 if p.startswith('/'):
9 return p
10 else:
11 return os.path.relpath(p, base)
12
13 Defaults.ConstructionEnvironment['_usw_relpath'] = _usw_relpath

Post by Hua Yanghao
Hi Bill,
I know it is a bad bad idea that's why I am asking for a better way :)
I almost forgot we have site_scons/site_init.py to play with. I create
quite a few different env but I guess if site_init.py is loaded fast
enough I should be able to override it there ...
Will let you know the result.
BR, Yanghao

Post by Hua Yanghao
correction: my test shows it is the *former*.

you should not have to change defaults.py (ever.. it's python you can reach into anything and change anything in your SConstruct/sconscript/etc)
So you want the string to be a/b/c.c and not build/a/b/c.c

Post by Bill Deegan
a/b/c.c
variant dir to
build/a/b/c.c
you're chdir'ing into
build/a/b ?
And you want the command line to be
compiler c.c
?

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

Jason Kenny

2018-10-24 17:05:18 UTC

Normally SCons will have the compiler output the target data into the Variant directory. In certain cases, the compiler doesn't allow control of where the output happens. SCons has a feature to change the current directory for you. However, this feature as it is, breaks the -j based builds. I work around this in my builds by changing the action to "cd <variant directory> ; <compiler command>". This seems to solve the issue of compilers messing up stuff on disk for me at least.
-Jason
________________________________
From: Scons-users <scons-users-***@scons.org> on behalf of Hua Yanghao <***@gmail.com>
Sent: Wednesday, October 17, 2018 2:55 PM
To: SCons users mailing list
Subject: [Scons-users] using variant_dir but strip out the variant_dir from source files

Dear Scons User,
I am having maybe an odd request. The background is there are some
compilers out there produces all kinds of output in your current
directory which you do not have control. I wanted to combine the
variant_dir feature plus the chdir kw arguments to builders, to be
able to invoke those compilers directly inside the variant_dir. And I
still want to keep the source duplication in the variant_dir as it
will be a snapshot of only the used files for a particular built out
of thousands of irrelevant files. As a side effect now the source
files are all prefixed with the variant_dir which becomes "invalid"
because the compiler is already invoked in the variant_dir instead of
the top level directory.

Is there a way (e.g. a special env variable) already that when I write
my builders instead of "$COMPILER $SOURCE", I can use something like
"$COMPILER $ORIGINAL_SOURCE"? Or do I have to wrap around the
$COMPILER with another command such that I can do things like
"$COMPILER_WRAPPED $VARIANT_DIR $SOURCE"? I know the second one
definitely can work but wondering if there are better ways already in
scons.

Best Regards,
Yanghao

Hua Yanghao

2018-10-24 19:59:44 UTC