[Scons-users] [Scons-dev] env.subst and variable substitution including whitespace in parameters

[Bill Deegan]

Did you try having pipes in your action command string?
I'm thinking it should work..

I wrote the wiki page you mentioned and it's a WIP.

BTW. This is really a scons-users mailing list item and not a scons-dev.

Hi,
I'm wondering if this should be filed as a github issue
Recently I ran into a problem, I needed to write a method to pipe some
input into and out of gcc via the preprocessor
this meant using subprocess instead of calling an scons Action since I
wanted to do piping, but to do that I needed to expand outwards the gcc /
cpppath etc into a string.
one approach to this might be
```
cmdstr = '${CPP} -E ${CFLAGS} ${CCFLAGS} ${_CCCOMCOM} -'
cmd = env.subst(cmdstr)
```
However if any of the include directories include spaces then this doesn't
work
``
arm-none-eabi-gcc -E -I. -ID:\Some Folder with whitespace\src -Ibuild -
```
``
arm-none-eabi-gcc -E -I. "-ID:\Some Folder with whitespace\src" -Ibuild -
```
It turns out the Action / CommandAction class has a way of dealing with
this.
the use of env.subst_list and the escape method burried in the
SCons/Subst.py code
Typically the function that puts quotes around strings contaning
whitespace is quote_spaces within SCons/Subst.py
If I follow the code back, this is called by CmdStringHolder which is only
referenced within the ListSubber class
which appears to be used by env.subst_list but not env.subst
The end result after a bit of copying / pasting is the below
I'm fairly sure this isn't using what's supposed to be public API's
```
env.AddMethod(PreProcessStream, 'PreProcessStream')
escape = env.get('ESCAPE', lambda x: x)
escape_list = SCons.Subst.escape_list
cmdstr = '${CPP} -E ${CFLAGS} ${CCFLAGS} ${_CCCOMCOM} -'
cmd_list = env.subst_list(cmdstr, SCons.Subst.SUBST_CMD)
cmd_list = escape_list(cmd_list[0], escape)
cmd = ' '.join(cmd_list)
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stdin=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate(input=stdin_str.encode())
return out, err
```
After a bit googling for "env.subst_list" it looks like a few others have
also used this method
I also came across this
* https://github.com/SCons/scons/wiki/ProposalForImprovedSubstEngine
So what I'm proposing here is that perhaps we need some public api way of
handling this problem without diving into the scons internals
Many Thanks
Richard
_______________________________________________
Scons-dev mailing list
https://pairlist2.pair.net/mailman/listinfo/scons-dev

[Mats Wichmann]

Post by Bill Deegan
Did you try having pipes in your action command string?
I'm thinking it should work..
I wrote the wiki page you mentioned and it's a WIP.

I'm wondering if this should be filed as a github issue
Recently I ran into a problem, I needed to write a method to pipe some
input into and out of gcc via the preprocessor
this meant using subprocess instead of calling an scons Action since I
wanted to do piping, but to do that I needed to expand outwards the gcc /
cpppath etc into a string.

there is some mechanism for piping in a command, see the PSPAWN
construction environment variable.

[Bill Deegan]

Really shouldn't need to run paths through subst to get the strings
properly escaped (or quoted depending on the platform)..

Can you create a small testcase demonstrating the problem, and I'll take a
look at it in a few days when I'm back from vacation..

Sorry I'll keep any replies to the user list,
I'm basically piping in a string from python then capturing the output and
using the results in python
so I'm capturing stdin and stdout. PSPAWN might work thanks for the info
I did also find another example where env.subst_list is useful
if you have a list of paths stored in an env variable, and those paths
contain spaces
```
env.Replace(Example1=['D:\\Temp\Test Folder 1\\src1.c'])
env.Append(Example1=['D:\\Temp\Test Folder 1\\src2.c'])
env.Append(Example1=['D:\\Temp\Test Folder 1\\src3.c'])
env.Replace(Example2=['D:\\Temp\Test Folder 1\\src4.c'])
env.Append(Example2=['D:\\Temp\Test Folder 1\\src5.c'])
env.Append(Example2=['D:\\Temp\Test Folder 1\\src6.c'])
env.Replace(SRCS='$Example1 $Example2')
# This doesn't work due to spaces in path
#items = env.Split('$SRCS')
# This results in a single string with unescaped paths
# making it difficult to split them apart or escape them
#items = env.subst('$SRCS')
items = env.subst_list('$SRCS')[0]
print(item)
```
this is why I was wondering if having env.subst_list as some form of
public api should be on the github issues / todo list
to give some context, this is what the Preprocess tool I've written looks
like so far
```
env.AddMethod(PreProcessStream, 'PreProcessStream')
# Handle escaping / quoting variable expansion
escape = env.get('ESCAPE', lambda x: x)
escape_list = SCons.Subst.escape_list
cmdstr = '${CPP} -E ${CFLAGS} ${CCFLAGS} ${_CCCOMCOM} -'
cmd_list = env.subst_list(cmdstr, SCons.Subst.SUBST_CMD)
cmd_list = escape_list(cmd_list[0], escape)
cmd = ' '.join(cmd_list)
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stdin=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate(input=stdin_str.encode())
return out, err
```
Many Thanks
Richard

Post by Mats Wichmann

Post by Bill Deegan
Did you try having pipes in your action command string?
I'm thinking it should work..
I wrote the wiki page you mentioned and it's a WIP.

I'm wondering if this should be filed as a github issue
Recently I ran into a problem, I needed to write a method to pipe some
input into and out of gcc via the preprocessor
this meant using subprocess instead of calling an scons Action since I
wanted to do piping, but to do that I needed to expand outwards the

gcc /

Post by Bill Deegan

cpppath etc into a string.

there is some mechanism for piping in a command, see the PSPAWN
construction environment variable.
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users