[Scons-users] File / Dir relative paths and SConscriptChdir

[Bill Deegan]

Perhaps you can explain what you're actually trying to do, rather than
providing a toy example?
That will help provide more context correct help.

Hi,
I've got another question, this is over the use of SConscriptChdir
such as env.SConscriptChdir(0)
from what I can gather when specifying a File or Dir object or making a
call to a Command builder at least which will automatically convert a
string target to a File object.
The File / Dir object will always be relative to the current SConstruct
file.
Not relative to the originating SConstruct file (which is what I'm trying
to do)
To give an example
src1\test1.txt - empty test file
src1\test2.txt - empty test file
src1\SConstruct
```
import os
EnsureSConsVersion(3, 0, 0)
# Setup the construction env
env = Environment(ENV={'PATH': os.environ['PATH']})
# Set working directory to not change
env.SConscriptChdir(0)
# Load in an external script
SConscript('../src2/testsubscript.py', {'env': env})
```
src2\SConstruct
```
# Import the scons environment from the calling script
Import('env')
# At this point the 'testfile.txt' will be automatically changed to a
File class by SCons
# print the path scons thinks this file is located at
print(target[0].path)
tgt = env.Command(File('testfile.txt'), [], build_test)
env.Default(tgt)
tgt2 = env.Command('testfile2.txt', [], build_test)
env.Default(tgt2)
# Even though SConscriptChdir is set to 0 in the originating env, File
uses the current script as it's reference point
# D:\Temp\testproject\src2\testfile.txt
# D:\Temp\testproject\src2\testfile2.txt
```
Is this expected behaviour?
perhaps SConscriptChdir only influences when calling exe's rather than the
use of File or Dir
is there some public API way of setting the current directory for File /
Dir calls?
thats not the directory of the current SConsscript
since builders wrap everything in a File automatically, it might mean I
have to make sure all file paths are absolute before hand.
Many Thanks
Richard
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

So much that could be improved with your code example below.
Firstly almost no reason to use File() or Dir(), just use a string.
Secondly no need to have File(subst...) use File("${TOP}/....")

And never have a builder with no sources. If it's no sources that means it
can be run when the SConstruct/SConscript is processed, so use Execute.
If it can't be run when SConstruct/SConscripts are processed, then it
depends on something, and you should list those as sources. Your builder
can pick and choose what of $SOURCES it uses (or nothing).

What directory does your SConstruct live in?

-Bill

I've sorted it out now so no worries, I've also removed the
SConscriptChdir() call
what I've basically got is
a SConscript within a directory such as
D:\Temp\micropython\ports\minimal
(there being multiple ports under different directories
This then pulls in a seperate common SConscript under a path such as
D:\Temp\micropython\tools\scons\micropy.py
via SConscript('../../tools/scons/micropy.py', {'env': env})
because the common script lives in a different directory I needed to make
sure the paths resolved correctly
Putting in a # at the beginning of the path seems to work best
this way it will resolve to D:\Temp\micropython\ports\minimal instead of
D:\Temp\micropython\tools\scons
which is where the top level script lives
As a simple extract
```
# TOP Resolves to D:\Temp\micropython
env.SetDefault(TOP=Dir('../..'))
env.SetDefault(BUILD='build')
env.Replace(HEADER_BUILD='${BUILD}/genhdr')
# Build the micropython version header
# Use File to make sure the directory seperator is the correct way
round
cmd = File(env.subst('${TOP}/py/makeversionhdr.py')).abspath
cmd += ' ' + target[0].path
env.Execute(cmd, chdir=env.GetLaunchDir())
tgt = env.Command('#${HEADER_BUILD}/mpversion.h', [], build_mpversion)
```
Note there's a lot more in this file than just this
and part of the initial exercise is to just try and keep the existing
logic / if statements
from the old Makefile intact to make sure it works before I start to
re-arrange stuff to clean it up
One thing I have noticed is that if you provide a construction variable as
part of the name to a File or Dir
such as File('${TOP}/py/makeversionhdr.py') then run .abspath on it for
the full path
it doesn't look like it expands out the construction variable
but this does work
env.Replace(TEST1=Dir('../..'))
env.Replace(TEST2='$TEST1\\additionaldir')
test1 = env.subst('$TEST2')
# test1 now equals D:\Temp\micropython\additionaldir

What command line are you seeing?
Get rid of your SConscriptChdir() call.
I never use it.
You can add as an argument to SConscript, but generally the default
behavior just works.
Chdir generally breaks parallel builds.
It's possible the default mentioned in the manpage is incorrect. As far
as I know all paths are specified from dir and chdir is only done if
explicitly specified to a builder..

So you have two builders with empty sources?
Not good practice.
Use Execute if you want it to always run..
-Bill

Hi Bill,
I think I've figured it out
basically I'm trying to use relative paths which are relative to the
top level SConstruct file
instead of being relative to a second called script in another directory
the first way around this is to use the directory parameter
tgt = env.Command(File('testfile.txt', directory ='../src1'), [],
build_test)
the second way around this is to prefix the path with #
tgt = env.Command(File('#testfile.txt'), [], build_test)

Post by Bill Deegan
Perhaps you can explain what you're actually trying to do, rather than
providing a toy example?
That will help provide more context correct help.
On Thu, Sep 6, 2018 at 4:23 PM RW via Scons-users <

Hi,
I've got another question, this is over the use of SConscriptChdir
such as env.SConscriptChdir(0)
from what I can gather when specifying a File or Dir object or making
a call to a Command builder at least which will automatically convert a
string target to a File object.
The File / Dir object will always be relative to the current
SConstruct file.
Not relative to the originating SConstruct file (which is what I'm
trying to do)
To give an example
src1\test1.txt - empty test file
src1\test2.txt - empty test file
src1\SConstruct
```
import os
EnsureSConsVersion(3, 0, 0)
# Setup the construction env
env = Environment(ENV={'PATH': os.environ['PATH']})
# Set working directory to not change
env.SConscriptChdir(0)
# Load in an external script
SConscript('../src2/testsubscript.py', {'env': env})
```
src2\SConstruct
```
# Import the scons environment from the calling script
Import('env')
# At this point the 'testfile.txt' will be automatically changed
to a File class by SCons
# print the path scons thinks this file is located at
print(target[0].path)
tgt = env.Command(File('testfile.txt'), [], build_test)
env.Default(tgt)
tgt2 = env.Command('testfile2.txt', [], build_test)
env.Default(tgt2)
# Even though SConscriptChdir is set to 0 in the originating env,
File uses the current script as it's reference point
# D:\Temp\testproject\src2\testfile.txt
# D:\Temp\testproject\src2\testfile2.txt
```
Is this expected behaviour?
perhaps SConscriptChdir only influences when calling exe's rather
than the use of File or Dir
is there some public API way of setting the current directory for
File / Dir calls?
thats not the directory of the current SConsscript
since builders wrap everything in a File automatically, it might mean
I have to make sure all file paths are absolute before hand.
Many Thanks
Richard
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________

Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

# You shouldn't need to do this. Python's open will handle either slash on
windows.
# header = env.subst('${HEADER_BUILD}/mpversion.h')
# header = os.path.normpath(header)

# you shouldn't need to do this as ${TOP}. will get expanded by execute
# script = env.subst('${TOP}/py/makeversionhdr.py')
# script = os.path.abspath(script)

# Why do you need chdir? Do you control the script?
env.Execute("'${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h",
chdir=env.GetLaunchDir())

-Bill

Hi Bill,
I've replaced File with os.path.normpath / os.path.abspath
(to make sure the slashes are the right way round for Windows or Linux)
and I'm now running it directly in the SConscript instead of as a builder
since there are no sources for this particular one
(but there will be for others)
```
header = env.subst('${HEADER_BUILD}/mpversion.h')
header = os.path.normpath(header)
script = env.subst('${TOP}/py/makeversionhdr.py')
script = os.path.abspath(script)
env.Execute(script + ' ' + header, chdir=env.GetLaunchDir())
```
I was using File() instead of env.File which is why the variable
substitution failed from the looks of things

Post by Bill Deegan
So much that could be improved with your code example below.
Firstly almost no reason to use File() or Dir(), just use a string.
Secondly no need to have File(subst...) use File("${TOP}/....")
And never have a builder with no sources. If it's no sources that means
it can be run when the SConstruct/SConscript is processed, so use Execute.
If it can't be run when SConstruct/SConscripts are processed, then it
depends on something, and you should list those as sources. Your builder
can pick and choose what of $SOURCES it uses (or nothing).
What directory does your SConstruct live in?
-Bill

I've sorted it out now so no worries, I've also removed the
SConscriptChdir() call
what I've basically got is
a SConscript within a directory such as
D:\Temp\micropython\ports\minimal
(there being multiple ports under different directories
This then pulls in a seperate common SConscript under a path such as
D:\Temp\micropython\tools\scons\micropy.py
via SConscript('../../tools/scons/micropy.py', {'env': env})
because the common script lives in a different directory I needed to
make sure the paths resolved correctly
Putting in a # at the beginning of the path seems to work best
this way it will resolve to D:\Temp\micropython\ports\minimal instead of
D:\Temp\micropython\tools\scons
which is where the top level script lives
As a simple extract
```
# TOP Resolves to D:\Temp\micropython
env.SetDefault(TOP=Dir('../..'))
env.SetDefault(BUILD='build')
env.Replace(HEADER_BUILD='${BUILD}/genhdr')
# Build the micropython version header
# Use File to make sure the directory seperator is the correct way
round
cmd = File(env.subst('${TOP}/py/makeversionhdr.py')).abspath
cmd += ' ' + target[0].path
env.Execute(cmd, chdir=env.GetLaunchDir())
tgt = env.Command('#${HEADER_BUILD}/mpversion.h', [], build_mpversion)
```
Note there's a lot more in this file than just this
and part of the initial exercise is to just try and keep the existing
logic / if statements
from the old Makefile intact to make sure it works before I start to
re-arrange stuff to clean it up
One thing I have noticed is that if you provide a construction variable
as part of the name to a File or Dir
such as File('${TOP}/py/makeversionhdr.py') then run .abspath on it for
the full path
it doesn't look like it expands out the construction variable
but this does work
env.Replace(TEST1=Dir('../..'))
env.Replace(TEST2='$TEST1\\additionaldir')
test1 = env.subst('$TEST2')
# test1 now equals D:\Temp\micropython\additionaldir

What command line are you seeing?
Get rid of your SConscriptChdir() call.
I never use it.
You can add as an argument to SConscript, but generally the default
behavior just works.
Chdir generally breaks parallel builds.
It's possible the default mentioned in the manpage is incorrect. As far
as I know all paths are specified from dir and chdir is only done if
explicitly specified to a builder..

So you have two builders with empty sources?
Not good practice.
Use Execute if you want it to always run..
-Bill

Hi Bill,
I think I've figured it out
basically I'm trying to use relative paths which are relative to the
top level SConstruct file
instead of being relative to a second called script in another directory
the first way around this is to use the directory parameter
tgt = env.Command(File('testfile.txt', directory ='../src1'), [],
build_test)
the second way around this is to prefix the path with #
tgt = env.Command(File('#testfile.txt'), [], build_test)

Post by Bill Deegan
Perhaps you can explain what you're actually trying to do, rather
than providing a toy example?
That will help provide more context correct help.
On Thu, Sep 6, 2018 at 4:23 PM RW via Scons-users <

Hi,
I've got another question, this is over the use of SConscriptChdir
such as env.SConscriptChdir(0)
from what I can gather when specifying a File or Dir object or
making a call to a Command builder at least which will automatically
convert a string target to a File object.
The File / Dir object will always be relative to the current
SConstruct file.
Not relative to the originating SConstruct file (which is what I'm
trying to do)
To give an example
src1\test1.txt - empty test file
src1\test2.txt - empty test file
src1\SConstruct
```
import os
EnsureSConsVersion(3, 0, 0)
# Setup the construction env
env = Environment(ENV={'PATH': os.environ['PATH']})
# Set working directory to not change
env.SConscriptChdir(0)
# Load in an external script
SConscript('../src2/testsubscript.py', {'env': env})
```
src2\SConstruct
```
# Import the scons environment from the calling script
Import('env')
# At this point the 'testfile.txt' will be automatically
changed to a File class by SCons
# print the path scons thinks this file is located at
print(target[0].path)
tgt = env.Command(File('testfile.txt'), [], build_test)
env.Default(tgt)
tgt2 = env.Command('testfile2.txt', [], build_test)
env.Default(tgt2)
# Even though SConscriptChdir is set to 0 in the originating env,
File uses the current script as it's reference point
# D:\Temp\testproject\src2\testfile.txt
# D:\Temp\testproject\src2\testfile2.txt
```
Is this expected behaviour?
perhaps SConscriptChdir only influences when calling exe's rather
than the use of File or Dir
is there some public API way of setting the current directory for
File / Dir calls?
thats not the directory of the current SConsscript
since builders wrap everything in a File automatically, it might
mean I have to make sure all file paths are absolute before hand.
Many Thanks
Richard
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________

Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________

Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

How about?

env.Execute([Mkdir("${TARGET.dir}"),'${TOP}/py/makeversionhdr.py
${HEADER_BUILD}/mpversion.h'])

forget I said that, it was because I was relying on the Command builder to
create the directory for the target
dooh

the reason for chdir is just because of the location of files
it makes porting the old Makefiles easier and just allows for relative
paths in the logged output making a bit easier to read
I basically have a common script sitting in one directory, and the top
level SConstruct sitting somewhere else
the layout of the files is partially dictated by the way the existing
source is setup
The variables do get expanded, but the windows slash isn't handled in the
executed script
possibly because I'm launching a different python script and it's not
handling the slashes in the parameter field
env.Execute('${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h',
chdir=env.GetLaunchDir())
D:\Temp\micropython/py/makeversionhdr.py "build/genhdr/mpversion.h"
GEN build/genhdr/mpversion.h
File "D:\Temp\micropython\py\makeversionhdr.py", line 107, in <module>
make_version_header(sys.argv[1])
File "D:\Temp\micropython\py\makeversionhdr.py", line 103, in
make_version_header
'build/genhdr/mpversion.h'

Post by Bill Deegan
# You shouldn't need to do this. Python's open will handle either slash
on windows.
# header = env.subst('${HEADER_BUILD}/mpversion.h')
# header = os.path.normpath(header)
# you shouldn't need to do this as ${TOP}. will get expanded by execute
# script = env.subst('${TOP}/py/makeversionhdr.py')
# script = os.path.abspath(script)
# Why do you need chdir? Do you control the script?
env.Execute("'${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h",
chdir=env.GetLaunchDir())
-Bill

Hi Bill,
I've replaced File with os.path.normpath / os.path.abspath
(to make sure the slashes are the right way round for Windows or Linux)
and I'm now running it directly in the SConscript instead of as a
builder since there are no sources for this particular one
(but there will be for others)
```
header = env.subst('${HEADER_BUILD}/mpversion.h')
header = os.path.normpath(header)
script = env.subst('${TOP}/py/makeversionhdr.py')
script = os.path.abspath(script)
env.Execute(script + ' ' + header, chdir=env.GetLaunchDir())
```
I was using File() instead of env.File which is why the variable
substitution failed from the looks of things

Post by Bill Deegan
So much that could be improved with your code example below.
Firstly almost no reason to use File() or Dir(), just use a string.
Secondly no need to have File(subst...) use File("${TOP}/....")
And never have a builder with no sources. If it's no sources that
means it can be run when the SConstruct/SConscript is processed, so use
Execute.
If it can't be run when SConstruct/SConscripts are processed, then it
depends on something, and you should list those as sources. Your builder
can pick and choose what of $SOURCES it uses (or nothing).
What directory does your SConstruct live in?
-Bill
On Thu, Sep 6, 2018 at 7:56 PM RW via Scons-users <

I've sorted it out now so no worries, I've also removed the
SConscriptChdir() call
what I've basically got is
a SConscript within a directory such as
D:\Temp\micropython\ports\minimal
(there being multiple ports under different directories
This then pulls in a seperate common SConscript under a path such as
D:\Temp\micropython\tools\scons\micropy.py
via SConscript('../../tools/scons/micropy.py', {'env': env})
because the common script lives in a different directory I needed to
make sure the paths resolved correctly
Putting in a # at the beginning of the path seems to work best
this way it will resolve to D:\Temp\micropython\ports\minimal instead
of D:\Temp\micropython\tools\scons
which is where the top level script lives
As a simple extract
```
# TOP Resolves to D:\Temp\micropython
env.SetDefault(TOP=Dir('../..'))
env.SetDefault(BUILD='build')
env.Replace(HEADER_BUILD='${BUILD}/genhdr')
# Build the micropython version header
# Use File to make sure the directory seperator is the correct
way round
cmd = File(env.subst('${TOP}/py/makeversionhdr.py')).abspath
cmd += ' ' + target[0].path
env.Execute(cmd, chdir=env.GetLaunchDir())
tgt = env.Command('#${HEADER_BUILD}/mpversion.h', [], build_mpversion)
```
Note there's a lot more in this file than just this
and part of the initial exercise is to just try and keep the existing
logic / if statements
from the old Makefile intact to make sure it works before I start to
re-arrange stuff to clean it up
One thing I have noticed is that if you provide a construction
variable as part of the name to a File or Dir
such as File('${TOP}/py/makeversionhdr.py') then run .abspath on it
for the full path
it doesn't look like it expands out the construction variable
but this does work
env.Replace(TEST1=Dir('../..'))
env.Replace(TEST2='$TEST1\\additionaldir')
test1 = env.subst('$TEST2')
# test1 now equals D:\Temp\micropython\additionaldir

What command line are you seeing?
Get rid of your SConscriptChdir() call.
I never use it.
You can add as an argument to SConscript, but generally the default
behavior just works.
Chdir generally breaks parallel builds.
It's possible the default mentioned in the manpage is incorrect. As
far as I know all paths are specified from dir and chdir is only done if
explicitly specified to a builder..
On Thu, Sep 6, 2018 at 5:58 PM Bill Deegan <

So you have two builders with empty sources?
Not good practice.
Use Execute if you want it to always run..
-Bill

Hi Bill,
I think I've figured it out
basically I'm trying to use relative paths which are relative to
the top level SConstruct file
instead of being relative to a second called script in another directory
the first way around this is to use the directory parameter
tgt = env.Command(File('testfile.txt', directory ='../src1'), [],
build_test)
the second way around this is to prefix the path with #
tgt = env.Command(File('#testfile.txt'), [], build_test)
On Thu, 6 Sep 2018 at 22:10, Bill Deegan <

Post by Bill Deegan
Perhaps you can explain what you're actually trying to do, rather
than providing a toy example?
That will help provide more context correct help.
On Thu, Sep 6, 2018 at 4:23 PM RW via Scons-users <

Hi,
I've got another question, this is over the use of
SConscriptChdir
such as env.SConscriptChdir(0)
from what I can gather when specifying a File or Dir object or
making a call to a Command builder at least which will automatically
convert a string target to a File object.
The File / Dir object will always be relative to the current
SConstruct file.
Not relative to the originating SConstruct file (which is what
I'm trying to do)
To give an example
src1\test1.txt - empty test file
src1\test2.txt - empty test file
src1\SConstruct
```
import os
EnsureSConsVersion(3, 0, 0)
# Setup the construction env
env = Environment(ENV={'PATH': os.environ['PATH']})
# Set working directory to not change
env.SConscriptChdir(0)
# Load in an external script
SConscript('../src2/testsubscript.py', {'env': env})
```
src2\SConstruct
```
# Import the scons environment from the calling script
Import('env')
# At this point the 'testfile.txt' will be automatically
changed to a File class by SCons
# print the path scons thinks this file is located at
print(target[0].path)
tgt = env.Command(File('testfile.txt'), [], build_test)
env.Default(tgt)
tgt2 = env.Command('testfile2.txt', [], build_test)
env.Default(tgt2)
# Even though SConscriptChdir is set to 0 in the originating
env, File uses the current script as it's reference point
# D:\Temp\testproject\src2\testfile.txt
# D:\Temp\testproject\src2\testfile2.txt
```
Is this expected behaviour?
perhaps SConscriptChdir only influences when calling exe's
rather than the use of File or Dir
is there some public API way of setting the current directory
for File / Dir calls?
thats not the directory of the current SConsscript
since builders wrap everything in a File automatically, it might
mean I have to make sure all file paths are absolute before hand.
Many Thanks
Richard
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________

Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________

Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________

Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users