[Scons-users] Creating SConscripts on the fly

[Spencer Yost]

I know this has been covered before, and I apologize for plowing old ground. I did search around but came up empty handed. I have a question:

I create my SConscript files on the fly. I have my own system for determining whether or not it needs to be built for a particular module in the tree, then I build it if needed and then make the SConscript() call. this works great. So I decided I want to create a builder for this SConscript creation - just because that seemed to be the "right" way to do it. But if I use a custom builder it will not build the software if it determines the SConscript file needs to be built. It just creates the SConscript file. If it doesn't have to create/recreate, it will build the software.

I have become experienced enough with Scons now to realize why it is doing this. But there has to be a way to get it to build the SConscript file using a builder and build the software without running a second time. Does anyone have a slick way of combining the creation of the SConscript while also building? I do not have a "root"(at the SConstruct level) SConscript file - which may the be the answer - but thought I would ask smarter people than me before I spent the time.

Thanks in advance!

Spencer Yost

[Gary Granger]

Hi Spencer,

If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and
let scons build the software normally, without requiring an intermediate
SConscript file?  What purpose does generating the SConscript file serve?

In normal operation, SCons first reads all the SConscript files to build
up the graph of nodes and dependencies, and then it builds whatever
nodes are outdated or missing.  So I don't think you can specify that a
SConscript file is a node that needs to be built, then read that
SConscript file to modify the graph of nodes and build again, not
without running SCons twice.

So in your code which determines whether a module needs to be built,
rather than generating a SConscript file, just call the builders that
the SConscript file would have called.  If a module does not need to be
built, then it is ok to skip calling those builders.  Once all the
builders for all the modules have been called, SCons will build all the
modules in one run.

I hope I understood your situation and this makes sense...
Gary

Post by Spencer Yost
I create my SConscript files on the fly. I have my own system for determining whether or not it needs to be built for a particular module in the tree, then I build it if needed and then make the SConscript() call. this works great. So I decided I want to create a builder for this SConscript creation - just because that seemed to be the "right" way to do it. But if I use a custom builder it will not build the software if it determines the SConscript file needs to be built. It just creates the SConscript file. If it doesn't have to create/recreate, it will build the software.
I have become experienced enough with Scons now to realize why it is doing this. But there has to be a way to get it to build the SConscript file using a builder and build the software without running a second time. Does anyone have a slick way of combining the creation of the SConscript while also building? I do not have a "root"(at the SConstruct level) SConscript file - which may the be the answer - but thought I would ask smarter people than me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Hill, Steve (FP COM)]

why not just call the builders directly and let scons build the software normally, without requiring an intermediate SConscript file?

This is exactly what I do. The SConstruct invokes our layer of business logic which, once it has figured out what needs to be built, calls the SCons Builders and releases SCons to do the build. This has allowed us to get all the SCons build goodness but with build times, in some cases, an order of magnitude quicker or more




S.





From: Scons-users [mailto:scons-users-***@scons.org] On Behalf Of Gary Granger
Sent: 22 February 2018 06:09
To: Spencer Yost; SCons users mailing list
Subject: Re: [Scons-users] Creating SConscripts on the fly



Hi Spencer,

If you already have the information to generate the SConscript file from within a scons builder, then why not just call the builders directly and let scons build the software normally, without requiring an intermediate SConscript file? What purpose does generating the SConscript file serve?

In normal operation, SCons first reads all the SConscript files to build up the graph of nodes and dependencies, and then it builds whatever nodes are outdated or missing. So I don't think you can specify that a SConscript file is a node that needs to be built, then read that SConscript file to modify the graph of nodes and build again, not without running SCons twice.

So in your code which determines whether a module needs to be built, rather than generating a SConscript file, just call the builders that the SConscript file would have called. If a module does not need to be built, then it is ok to skip calling those builders. Once all the builders for all the modules have been called, SCons will build all the modules in one run.

I hope I understood your situation and this makes sense...
Gary

On 02/21/2018 10:35 PM, Spencer Yost wrote:

I know this has been covered before, and I apologize for plowing old ground. I did search around but came up empty handed. I have a question:

I create my SConscript files on the fly. I have my own system for determining whether or not it needs to be built for a particular module in the tree, then I build it if needed and then make the SConscript() call. this works great. So I decided I want to create a builder for this SConscript creation - just because that seemed to be the "right" way to do it. But if I use a custom builder it will not build the software if it determines the SConscript file needs to be built. It just creates the SConscript file. If it doesn't have to create/recreate, it will build the software.

I have become experienced enough with Scons now to realize why it is doing this. But there has to be a way to get it to build the SConscript file using a builder and build the software without running a second time. Does anyone have a slick way of combining the creation of the SConscript while also building? I do not have a "root"(at the SConstruct level) SConscript file - which may the be the answer - but thought I would ask smarter people than me before I spent the time.

Thanks in advance!

Spencer Yost

_______________________________________________
Scons-users mailing list
Scons-***@scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Hua Yanghao]

One slight drawback of this approach I see is: you lost the out of box
variant_dir parameter support.

Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.

I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and let
scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build up
the graph of nodes and dependencies, and then it builds whatever nodes are
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript file to
modify the graph of nodes and build again, not without running SCons twice.
So in your code which determines whether a module needs to be built, rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the builders for all
the modules have been called, SCons will build all the modules in one run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing old ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to do
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is doing
this. But there has to be a way to get it to build the SConscript file using
a builder and build the software without running a second time. Does anyone
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level) SConscript file -
which may the be the answer - but thought I would ask smarter people than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.

-Bill

Post by Hua Yanghao
One slight drawback of this approach I see is: you lost the out of box
variant_dir parameter support.
Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.
I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and

let

Post by Gary Granger
scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build

up

Post by Gary Granger
the graph of nodes and dependencies, and then it builds whatever nodes

are

Post by Gary Granger
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript file to
modify the graph of nodes and build again, not without running SCons

twice.

Post by Gary Granger
So in your code which determines whether a module needs to be built,

rather

Post by Gary Granger
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be

built,

Post by Gary Granger
then it is ok to skip calling those builders. Once all the builders for

all

Post by Gary Granger
the modules have been called, SCons will build all the modules in one

run.

Post by Gary Granger
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing old

ground.

Post by Gary Granger
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module

in

Post by Gary Granger
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to

do

Post by Gary Granger
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is

doing

Post by Gary Granger
this. But there has to be a way to get it to build the SConscript file

using

Post by Gary Granger
a builder and build the software without running a second time. Does

anyone

Post by Gary Granger
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level) SConscript

file -

Post by Gary Granger
which may the be the answer - but thought I would ask smarter people

than

Post by Gary Granger
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Jason Kenny]

I agree. There are some cases that might be a little trick to get right if you have sources that are not under the sconstruct/sconscript that refers to the files. Also one tip that I have is to make sure your root path to the variantdir starts with a ‘#’ this will help prevent issues as your call down the tree into different build files.

I have seen other people use SCons more as an API. Since you seem to be saying that they are not going to call SCons to do the build and you don’t seem to be doing a automake(scons) like generator that makes sconstructs that you can call. You might want to think of SCons more as an API. I new on team that for some reason thought XML was the future of everything and they made a build system that read xml files and called scons to define what was to be built. Scons here was just an API. My system Parts tries to extent scons, but I tend to do stuff to scons at an API level to make this happen. I also track a component tree ( vs a node tree) help me tell if a component needs to be built or not and if not is don’t load the build files to help speed up the builds. I have plans to allow a “part” to load other type of files and parse those files and define to SCons what to do. It look you are wanting to do something similar. I have not done the builder generator way you talk about as this was tricky in how to handle the emitted file ( ie what it should build) to Scons. I personally wanted to make sure that rebuild are fast and correct. Doing what you suggest it not hard for a first pass build, but defining everything so a second pass only build what is needed correctly is the tricky piece here, I am 100% sure you need to define some scanner to make this work right.. I believe Scons supports the idea of new node tree added during build time, however I do not have a good example of how to do this correctly. I think this is what you need to make this work 100% correctly.

Bill do we have an example of such as thing? Ie I think two cases could useful here:


1. A builder/scanner example that allows one to add a whole new build tree? Ie the scanner reads you file and emits these files as a depends ( maybe also call some builders to define how to build some of the files)
2. A builder that does some work and at the end can scan and add to the node tree what it did. ( This is useful for cases when all we know if that this file will make some stuff. But we don’t know what it is until it is done and we scan a directory for certain types of files) I am not sure if this case is even possible, but knowing that if the other “build” file did not change means we know we should call it if one the nodes changed, and if the other build file did change we want to call it anyways as it probably changed what it will output and the node tree needs to be updated.

I think if we had example of these cases ( if it is possible) would make what is he would like to do much easier and would help in defining other builders ( such as a builders that call other build systems or creating documentation builder a lot easier to define and be correct on rebuilding when it should)

Jason

From: Scons-users [mailto:scons-users-***@scons.org] On Behalf Of Bill Deegan
Sent: Thursday, February 22, 2018 7:46 AM
To: SCons users mailing list <scons-***@scons.org>
Subject: Re: [Scons-users] Creating SConscripts on the fly

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.
-Bill

On Thu, Feb 22, 2018 at 4:28 AM, Hua Yanghao <***@gmail.com<mailto:***@gmail.com>> wrote:
One slight drawback of this approach I see is: you lost the out of box
variant_dir parameter support.

Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.

I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and let
scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build up
the graph of nodes and dependencies, and then it builds whatever nodes are
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript file to
modify the graph of nodes and build again, not without running SCons twice.
So in your code which determines whether a module needs to be built, rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the builders for all
the modules have been called, SCons will build all the modules in one run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing old ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to do
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is doing
this. But there has to be a way to get it to build the SConscript file using
a builder and build the software without running a second time. Does anyone
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level) SConscript file -
which may the be the answer - but thought I would ask smarter people than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C21e37316701e4614b1d108d579fa962f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636549039519315262&sdata=sHHWGkb7CuXVIfIYnRbyTFsuEfT%2FkNGwh50OZPMEVfE%3D&reserved=0>
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C21e37316701e4614b1d108d579fa962f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636549039519315262&sdata=sHHWGkb7CuXVIfIYnRbyTFsuEfT%2FkNGwh50OZPMEVfE%3D&reserved=0>

_______________________________________________
Scons-users mailing list
Scons-***@scons.org<mailto:Scons-***@scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C21e37316701e4614b1d108d579fa962f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636549039519315262&sdata=sHHWGkb7CuXVIfIYnRbyTFsuEfT%2FkNGwh50OZPMEVfE%3D&reserved=0>

[Hua Yanghao]

@Bill, my understanding is if you use VariantDir() and when building
the object list you will have to add the build path for each and every
one of them? as compared if the object list is directly returned from
a SConscript() call with variant_dir parameter then the object
location is already in the build folder. I think you have even give an

Here's two ways which are similar to using variant_dir in a SConscript call.
env.Program('some_build_dir/program', ['src/a.c','src/b.c'])
Assuming you have duplicate = 0
Or you can specify the
VariantDir('some_build_dir','src')
env.Program('some_build_dir/program', [os.path.join('some_build_dir',a) for a in ['a.c','b.c'])

Best Regards,
Yanghao

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.
-Bill

Post by Hua Yanghao
One slight drawback of this approach I see is: you lost the out of box
variant_dir parameter support.
Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.
I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and let
scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build up
the graph of nodes and dependencies, and then it builds whatever nodes are
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript file to
modify the graph of nodes and build again, not without running SCons twice.
So in your code which determines whether a module needs to be built, rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the builders for all
the modules have been called, SCons will build all the modules in one run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing old ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to do
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is doing
this. But there has to be a way to get it to build the SConscript file using
a builder and build the software without running a second time. Does anyone
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level) SConscript file -
which may the be the answer - but thought I would ask smarter people than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

Yanghao,

Yes that's correct.
That's what we used to do before variant_dir was added to SConscript.

But not entirely complete.

An initial *#* (hash mark) on a path name means that the rest of the file
name is interpreted relative to the directory containing the top-level
*SConstruct* file, even if the *#* is followed by a directory separator
character (slash or backslash).

You can reference the "current dir" via #

So:
VariantDir('build/src/a","src/a")
SConscript("build/src/a/SConscript",...)

Then all local references don't need to use build/src/a, but can use #

-Bill

Post by Hua Yanghao
@Bill, my understanding is if you use VariantDir() and when building
the object list you will have to add the build path for each and every
one of them? as compared if the object list is directly returned from
a SConscript() call with variant_dir parameter then the object
location is already in the build folder. I think you have even give an

Here's two ways which are similar to using variant_dir in a SConscript

call.

env.Program('some_build_dir/program', ['src/a.c','src/b.c'])
Assuming you have duplicate = 0
Or you can specify the
VariantDir('some_build_dir','src')
env.Program('some_build_dir/program', [os.path.join('some_build_dir',a)

for a in ['a.c','b.c'])
Best Regards,
Yanghao

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.
-Bill

Post by Hua Yanghao
One slight drawback of this approach I see is: you lost the out of box
variant_dir parameter support.
Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.
I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file

from

Post by Hua Yanghao

Post by Gary Granger
within a scons builder, then why not just call the builders directly

and

Post by Hua Yanghao

Post by Gary Granger
let
scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to

build

Post by Hua Yanghao

Post by Gary Granger
up
the graph of nodes and dependencies, and then it builds whatever nodes are
outdated or missing. So I don't think you can specify that a

SConscript

Post by Hua Yanghao

Post by Gary Granger
file is a node that needs to be built, then read that SConscript file

to

Post by Hua Yanghao

Post by Gary Granger
modify the graph of nodes and build again, not without running SCons twice.
So in your code which determines whether a module needs to be built, rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the builders

for

Post by Hua Yanghao

Post by Gary Granger
all
the modules have been called, SCons will build all the modules in one run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing old ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular

module

Post by Hua Yanghao

Post by Gary Granger
in
the tree, then I build it if needed and then make the SConscript()

call.

Post by Hua Yanghao

Post by Gary Granger
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way

to

Post by Hua Yanghao

Post by Gary Granger
do
it. But if I use a custom builder it will not build the software if

it

Post by Hua Yanghao

Post by Gary Granger
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is doing
this. But there has to be a way to get it to build the SConscript file using
a builder and build the software without running a second time. Does anyone
have a slick way of combining the creation of the SConscript while

also

Post by Hua Yanghao

Post by Gary Granger
building? I do not have a "root"(at the SConstruct level) SConscript file -
which may the be the answer - but thought I would ask smarter people than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Hua Yanghao]

Thanks for the tip Bill.

Wouldn't it be good that after VariantDir(), all subsequent calls will
assume a current working directory of the "build/src/a" folder
already?

My original confusion comes from: after VariantDir() call, I actually
want to setup a variant_dir for the entire source tree that has many
SConscript files (why would I need to setup a variant_dir for every
single SConscript anyway ...).
And however, I have to call SConscript on the variant_dir SConscript
file instead of the original SConscript file which was driving me
crazy back then. Any hints on why this is designed this way?

Best Regards,
Yanghao

Post by Bill Deegan
Yanghao,
Yes that's correct.
That's what we used to do before variant_dir was added to SConscript.
But not entirely complete.
An initial # (hash mark) on a path name means that the rest of the file
name is interpreted relative to the directory containing the top-level
SConstruct file, even if the # is followed by a directory separator
character (slash or backslash).
You can reference the "current dir" via #
VariantDir('build/src/a","src/a")
SConscript("build/src/a/SConscript",...)
Then all local references don't need to use build/src/a, but can use #
-Bill

Post by Hua Yanghao
@Bill, my understanding is if you use VariantDir() and when building
the object list you will have to add the build path for each and every
one of them? as compared if the object list is directly returned from
a SConscript() call with variant_dir parameter then the object
location is already in the build folder. I think you have even give an

Here's two ways which are similar to using variant_dir in a SConscript call.
env.Program('some_build_dir/program', ['src/a.c','src/b.c'])
Assuming you have duplicate = 0
Or you can specify the
VariantDir('some_build_dir','src')
env.Program('some_build_dir/program', [os.path.join('some_build_dir',a)
for a in ['a.c','b.c'])

Best Regards,
Yanghao

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.
-Bill

Post by Hua Yanghao
One slight drawback of this approach I see is: you lost the out of box
variant_dir parameter support.
Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.
I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and
let
scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build
up
the graph of nodes and dependencies, and then it builds whatever nodes
are
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript file to
modify the graph of nodes and build again, not without running SCons twice.
So in your code which determines whether a module needs to be built, rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the builders for
all
the modules have been called, SCons will build all the modules in one run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing old ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module
in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to
do
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is doing
this. But there has to be a way to get it to build the SConscript file
using
a builder and build the software without running a second time. Does anyone
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level) SConscript file -
which may the be the answer - but thought I would ask smarter people than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

Post by Hua Yanghao
Thanks for the tip Bill.
Wouldn't it be good that after VariantDir(), all subsequent calls will
assume a current working directory of the "build/src/a" folder
already?

No.
You may choose to have several VariantDir()'s grouped together in your
file, plus that would be implicit behavior.. (not so Pythonic)

Post by Hua Yanghao
My original confusion comes from: after VariantDir() call, I actually
want to setup a variant_dir for the entire source tree that has many
SConscript files (why would I need to setup a variant_dir for every
single SConscript anyway ...).

You don't have to.

Post by Hua Yanghao
And however, I have to call SConscript on the variant_dir SConscript
file instead of the original SConscript file which was driving me
crazy back then. Any hints on why this is designed this way?

Because you can have N variant_dirs for a given source dir allowing
multiple variations of a build to run at the same time.
For example {32,64}/{release/debug/profile}/{linux,win32,osx} could
potentially all build at the same time.

So with VariantDir (or variant_dir), you're saying build is a super
lightweight copy of src. And in most cases either only copy what you use,
or copy nothing, but act as if the source files were there.

Post by Hua Yanghao

Post by Bill Deegan
Yanghao,
Yes that's correct.
That's what we used to do before variant_dir was added to SConscript.
But not entirely complete.
An initial # (hash mark) on a path name means that the rest of the file
name is interpreted relative to the directory containing the top-level
SConstruct file, even if the # is followed by a directory separator
character (slash or backslash).
You can reference the "current dir" via #
VariantDir('build/src/a","src/a")
SConscript("build/src/a/SConscript",...)
Then all local references don't need to use build/src/a, but can use #
-Bill

Post by Hua Yanghao
@Bill, my understanding is if you use VariantDir() and when building
the object list you will have to add the build path for each and every
one of them? as compared if the object list is directly returned from
a SConscript() call with variant_dir parameter then the object
location is already in the build folder. I think you have even give an

Here's two ways which are similar to using variant_dir in a SConscript call.
env.Program('some_build_dir/program', ['src/a.c','src/b.c'])
Assuming you have duplicate = 0
Or you can specify the
VariantDir('some_build_dir','src')
env.Program('some_build_dir/program', [os.path.join('some_build_dir'

,a)

Post by Bill Deegan

Post by Hua Yanghao

for a in ['a.c','b.c'])

Best Regards,
Yanghao

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.
-Bill

Post by Hua Yanghao
One slight drawback of this approach I see is: you lost the out of

box

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao
variant_dir parameter support.
Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.
I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders

directly

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
and
let
scons build the software normally, without requiring an

intermediate

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build
up
the graph of nodes and dependencies, and then it builds whatever nodes
are
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript

file

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
to
modify the graph of nodes and build again, not without running

SCons

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
twice.
So in your code which determines whether a module needs to be

built,

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the

builders

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
for
all
the modules have been called, SCons will build all the modules in

one

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing

old

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module
in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right"

way

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
to
do
it. But if I use a custom builder it will not build the software

if

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will

build

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
the
software.
I have become experienced enough with Scons now to realize why it

is

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
doing
this. But there has to be a way to get it to build the SConscript file
using
a builder and build the software without running a second time.

Does

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
anyone
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level)

SConscript

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
file -
which may the be the answer - but thought I would ask smarter

people

Post by Bill Deegan

Post by Hua Yanghao

Post by Hua Yanghao

Post by Gary Granger
than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Hua Yanghao]

Because you can have N variant_dirs for a given source dir allowing multiple
variations of a build to run at the same time.
For example {32,64}/{release/debug/profile}/{linux,win32,osx} could
potentially all build at the same time.
So with VariantDir (or variant_dir), you're saying build is a super
lightweight copy of src. And in most cases either only copy what you use,
or copy nothing, but act as if the source files were there.

I see the fundamental difference here, I always wanted something like
build/64bit/... and build/32bit/... compared to build/module_x/{64bit,32bit}/...

But Indeed I see the point is valid.

Yes the copy on GNU/Linux is actually just a hard link, you can even
just modified the "copied" file which is the same as the original one.

[Bill Deegan]

You can do build/32 build/64

You don't have to do module_x level.

Post by Bill Deegan

Post by Bill Deegan
Because you can have N variant_dirs for a given source dir allowing

multiple

Post by Bill Deegan
variations of a build to run at the same time.
For example {32,64}/{release/debug/profile}/{linux,win32,osx} could
potentially all build at the same time.
So with VariantDir (or variant_dir), you're saying build is a super
lightweight copy of src. And in most cases either only copy what you

use,

Post by Bill Deegan
or copy nothing, but act as if the source files were there.

I see the fundamental difference here, I always wanted something like
build/64bit/... and build/32bit/... compared to
build/module_x/{64bit,32bit}/...
But Indeed I see the point is valid.
Yes the copy on GNU/Linux is actually just a hard link, you can even
just modified the "copied" file which is the same as the original one.
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Jason Kenny]

Post by Bill Deegan

So with VariantDir (or variant_dir), you're saying build is a super lightweight copy of src. And in most cases either only copy what you use, or copy nothing, but act as if the source files were there.

Just a quick note.. this only “copies” what SCons thinks it need to build. If you or Scons miss something the build can fail. I have also had issues with it correctly updating some files when it copies files vs making a hard link. System like Windows SCons and certain “remote” file system on lLnux will always copy.
You may want to use the “duplicate=False” argument in the SConscript() function. This has prevented a large number of issues for me. It also speeds up the build a little.
Jason


From: Scons-users [mailto:scons-users-***@scons.org] On Behalf Of Bill Deegan
Sent: Friday, February 23, 2018 2:06 PM
To: SCons users mailing list <scons-***@scons.org>
Subject: Re: [Scons-users] Creating SConscripts on the fly



On Fri, Feb 23, 2018 at 1:56 PM, Hua Yanghao <***@gmail.com<mailto:***@gmail.com>> wrote:
Thanks for the tip Bill.

Wouldn't it be good that after VariantDir(), all subsequent calls will
assume a current working directory of the "build/src/a" folder
already?

No.
You may choose to have several VariantDir()'s grouped together in your file, plus that would be implicit behavior.. (not so Pythonic)


My original confusion comes from: after VariantDir() call, I actually
want to setup a variant_dir for the entire source tree that has many
SConscript files (why would I need to setup a variant_dir for every
single SConscript anyway ...).

You don't have to.

And however, I have to call SConscript on the variant_dir SConscript
file instead of the original SConscript file which was driving me
crazy back then. Any hints on why this is designed this way?

Because you can have N variant_dirs for a given source dir allowing multiple variations of a build to run at the same time.
For example {32,64}/{release/debug/profile}/{linux,win32,osx} could potentially all build at the same time.
So with VariantDir (or variant_dir), you're saying build is a super lightweight copy of src. And in most cases either only copy what you use, or copy nothing, but act as if the source files were there.

Post by Bill Deegan
Yanghao,
Yes that's correct.
That's what we used to do before variant_dir was added to SConscript.
But not entirely complete.
An initial # (hash mark) on a path name means that the rest of the file
name is interpreted relative to the directory containing the top-level
SConstruct file, even if the # is followed by a directory separator
character (slash or backslash).
You can reference the "current dir" via #
VariantDir('build/src/a","src/a")
SConscript("build/src/a/SConscript",...)
Then all local references don't need to use build/src/a, but can use #
-Bill

@Bill, my understanding is if you use VariantDir() and when building
the object list you will have to add the build path for each and every
one of them? as compared if the object list is directly returned from
a SConscript() call with variant_dir parameter then the object
location is already in the build folder. I think you have even give an

Here's two ways which are similar to using variant_dir in a SConscript call.
env.Program('some_build_dir/program', ['src/a.c','src/b.c'])
Assuming you have duplicate = 0
Or you can specify the
VariantDir('some_build_dir','src')
env.Program('some_build_dir/program', [os.path.join('some_build_dir',a)
for a in ['a.c','b.c'])

Best Regards,
Yanghao

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.
-Bill

Post by Hua Yanghao
One slight drawback of this approach I see is: you lost the out of box
variant_dir parameter support.
Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.
I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and
let
scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build
up
the graph of nodes and dependencies, and then it builds whatever nodes
are
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript file to
modify the graph of nodes and build again, not without running SCons twice.
So in your code which determines whether a module needs to be built, rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the builders for
all
the modules have been called, SCons will build all the modules in one run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing old ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module
in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to
do
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is doing
this. But there has to be a way to get it to build the SConscript file
using
a builder and build the software without running a second time. Does anyone
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level) SConscript file -
which may the be the answer - but thought I would ask smarter people than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

_______________________________________________
Scons-users mailing list
Scons-***@scons.org<mailto:Scons-***@scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

[Bill Deegan]

If it's missing files, it means that your SConstruct logic is incomplete
and missing dependencies and/or outputs and you should fix that..

But yes duplicate=0 will be faster (the amount depending on the speed and
type of filesystem you're using)

Post by Bill Deegan

Post by Bill Deegan

Post by Bill Deegan
So with VariantDir (or variant_dir), you're saying build is a super

lightweight copy of src. And in most cases either only copy what you use,
or copy nothing, but act as if the source files were there.
Just a quick note.. this only “copies” what SCons thinks it need to build.
If you or Scons miss something the build can fail. I have also had issues
with it correctly updating some files when it copies files vs making a hard
link. System like Windows SCons and certain “remote” file system on lLnux
will always copy.
You may want to use the “duplicate=False” argument in the SConscript()
function. This has prevented a large number of issues for me. It also
speeds up the build a little.
Jason
Deegan
*Sent:* Friday, February 23, 2018 2:06 PM
*Subject:* Re: [Scons-users] Creating SConscripts on the fly
Thanks for the tip Bill.
Wouldn't it be good that after VariantDir(), all subsequent calls will
assume a current working directory of the "build/src/a" folder
already?
No.
You may choose to have several VariantDir()'s grouped together in your
file, plus that would be implicit behavior.. (not so Pythonic)
My original confusion comes from: after VariantDir() call, I actually
want to setup a variant_dir for the entire source tree that has many
SConscript files (why would I need to setup a variant_dir for every
single SConscript anyway ...).
You don't have to.
And however, I have to call SConscript on the variant_dir SConscript
file instead of the original SConscript file which was driving me
crazy back then. Any hints on why this is designed this way?
Because you can have N variant_dirs for a given source dir allowing
multiple variations of a build to run at the same time.
For example {32,64}/{release/debug/profile}/{linux,win32,osx} could
potentially all build at the same time.
So with VariantDir (or variant_dir), you're saying build is a super
lightweight copy of src. And in most cases either only copy what you use,
or copy nothing, but act as if the source files were there.

Post by Bill Deegan
Yanghao,
Yes that's correct.
That's what we used to do before variant_dir was added to SConscript.
But not entirely complete.
An initial # (hash mark) on a path name means that the rest of the file
name is interpreted relative to the directory containing the top-level
SConstruct file, even if the # is followed by a directory separator
character (slash or backslash).
You can reference the "current dir" via #
VariantDir('build/src/a","src/a")
SConscript("build/src/a/SConscript",...)
Then all local references don't need to use build/src/a, but can use #
-Bill

Post by Bill Deegan
@Bill, my understanding is if you use VariantDir() and when building
the object list you will have to add the build path for each and every
one of them? as compared if the object list is directly returned from
a SConscript() call with variant_dir parameter then the object
location is already in the build folder. I think you have even give an

Here's two ways which are similar to using variant_dir in a SConscript call.
env.Program('some_build_dir/program', ['src/a.c','src/b.c'])
Assuming you have duplicate = 0
Or you can specify the
VariantDir('some_build_dir','src')
env.Program('some_build_dir/program', [os.path.join('some_build_dir'

,a)

Post by Bill Deegan

Post by Bill Deegan

for a in ['a.c','b.c'])

Best Regards,
Yanghao

variant_dir is just a short cut for VariantDir and SConscript.
You don't loose anything.
-Bill

Post by Hua Yanghao
One slight drawback of this approach I see is: you lost the out of

box

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao
variant_dir parameter support.
Bill did show a way in another threads how this can be done also
directly using VariantDir(), so I think this slight drawback is
solvable.
I am thinking this is really a way to solve my SConscript
"default_import" issue, however this means also SConscript will be
hiding from module developers. But I think this is not a problem for
me.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders

directly

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
and
let
scons build the software normally, without requiring an

intermediate

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build
up
the graph of nodes and dependencies, and then it builds whatever nodes
are
outdated or missing. So I don't think you can specify that a SConscript
file is a node that needs to be built, then read that SConscript

file

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
to
modify the graph of nodes and build again, not without running

SCons

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
twice.
So in your code which determines whether a module needs to be

built,

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
rather
than generating a SConscript file, just call the builders that the
SConscript file would have called. If a module does not need to be built,
then it is ok to skip calling those builders. Once all the

builders

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
for
all
the modules have been called, SCons will build all the modules in

one

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
run.
I hope I understood your situation and this makes sense...
Gary
I know this has been covered before, and I apologize for plowing

old

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
ground.
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module
in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right"

way

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
to
do
it. But if I use a custom builder it will not build the software

if

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will

build

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
the
software.
I have become experienced enough with Scons now to realize why it

is

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
doing
this. But there has to be a way to get it to build the SConscript file
using
a builder and build the software without running a second time.

Does

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
anyone
have a slick way of combining the creation of the SConscript while also
building? I do not have a "root"(at the SConstruct level)

SConscript

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
file -
which may the be the answer - but thought I would ask smarter

people

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
than
me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao

Post by Gary Granger
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

Post by Bill Deegan

Post by Bill Deegan

Post by Hua Yanghao
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

Post by Bill Deegan

Post by Bill Deegan

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

Post by Bill Deegan

Post by Bill Deegan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>

Post by Bill Deegan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users
<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C572f50c20e5d44db4a2308d57af8e201%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636550131712437897&sdata=ZMennxPWvkWhZ3u3TMTF577JNHkqqn6NhM0uhtMcL8M%3D&reserved=0>
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[s***@triad.rr.com]

Thanks - we are converting to SCons, and the powers that be indicated that the developers would only maintain the old style build configuration files. So I have to parse them and build on the fly based on their contents. To prevent confusing this conversation with the "whys", lets just say for the moment creating SConscript files on the fly is necessary (explanation below if you care).

So I guess my question is, restated more clearly: Can I use SCons to build SConscript files and process them in one pass? Based on my limited experience and the replies it seems like there is not; but I figured maybe someone has this figured out.

Explanation: I could go ahead and simply execute the statements that I currently store in an SConscript; inline during the SConsctruct processing. But that creates a couple of dilemmas for us, mainly no "audit trail". The SConscript serves as a great, succinct audit of what a build would do for a particular module and aids in unit tests for, and code review of, my work. That was especially handy early on while testing the parsing. Also - I am hoping when they see how simple the SConscript files really are, we will finally get rid of the old config files and start maintaining the SConscript files. In the end, the building inline with SConstruct is a nut I am not interested in cracking right now.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and
let scons build the software normally, without requiring an intermediate
SConscript file?  What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build
up the graph of nodes and dependencies, and then it builds whatever
nodes are outdated or missing.  So I don't think you can specify that a
SConscript file is a node that needs to be built, then read that
SConscript file to modify the graph of nodes and build again, not
without running SCons twice.
So in your code which determines whether a module needs to be built,
rather than generating a SConscript file, just call the builders that
the SConscript file would have called.  If a module does not need to be
built, then it is ok to skip calling those builders.  Once all the
builders for all the modules have been called, SCons will build all the
modules in one run.
I hope I understood your situation and this makes sense...
Gary

Post by Spencer Yost
I create my SConscript files on the fly. I have my own system for determining whether or not it needs to be built for a particular module in the tree, then I build it if needed and then make the SConscript() call. this works great. So I decided I want to create a builder for this SConscript creation - just because that seemed to be the "right" way to do it. But if I use a custom builder it will not build the software if it determines the SConscript file needs to be built. It just creates the SConscript file. If it doesn't have to create/recreate, it will build the software.
I have become experienced enough with Scons now to realize why it is doing this. But there has to be a way to get it to build the SConscript file using a builder and build the software without running a second time. Does anyone have a slick way of combining the creation of the SConscript while also building? I do not have a "root"(at the SConstruct level) SConscript file - which may the be the answer - but thought I would ask smarter people than me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

Post by s***@triad.rr.com
Thanks - we are converting to SCons, and the powers that be indicated that
the developers would only maintain the old style build configuration
files. So I have to parse them and build on the fly based on their
contents. To prevent confusing this conversation with the "whys", lets
just say for the moment creating SConscript files on the fly is necessary
(explanation below if you care).
So I guess my question is, restated more clearly: Can I use SCons to
build SConscript files and process them in one pass? Based on my limited
experience and the replies it seems like there is not; but I figured maybe
someone has this figured out.

I'd say.. Sort of.. See my previous answer in this thread. Do the
conversion in SConstruct, but not using SCons' dependency tree. I'm
guessing the build files don't have any dependencies? If so, that it should
be fairly simple. The tree's utility goes up when you have complicated
sets of dependencies.

Post by s***@triad.rr.com
Explanation: I could go ahead and simply execute the statements that I
currently store in an SConscript; inline during the SConsctruct
processing. But that creates a couple of dilemmas for us, mainly no "audit
trail". The SConscript serves as a great, succinct audit of what a build
would do for a particular module and aids in unit tests for, and code
review of, my work. That was especially handy early on while testing the
parsing. Also - I am hoping when they see how simple the SConscript files
really are, we will finally get rid of the old config files and start
maintaining the SConscript files. In the end, the building inline with
SConstruct is a nut I am not interested in cracking right now.

Post by Gary Granger
Hi Spencer,
If you already have the information to generate the SConscript file from
within a scons builder, then why not just call the builders directly and
let scons build the software normally, without requiring an intermediate
SConscript file? What purpose does generating the SConscript file serve?
In normal operation, SCons first reads all the SConscript files to build
up the graph of nodes and dependencies, and then it builds whatever
nodes are outdated or missing. So I don't think you can specify that a
SConscript file is a node that needs to be built, then read that
SConscript file to modify the graph of nodes and build again, not
without running SCons twice.
So in your code which determines whether a module needs to be built,
rather than generating a SConscript file, just call the builders that
the SConscript file would have called. If a module does not need to be
built, then it is ok to skip calling those builders. Once all the
builders for all the modules have been called, SCons will build all the
modules in one run.
I hope I understood your situation and this makes sense...
Gary

Post by Spencer Yost
I know this has been covered before, and I apologize for plowing old
I create my SConscript files on the fly. I have my own system for

determining whether or not it needs to be built for a particular module in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to do
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.

Post by Gary Granger

Post by Spencer Yost
I have become experienced enough with Scons now to realize why it is

doing this. But there has to be a way to get it to build the SConscript
file using a builder and build the software without running a second time.
Does anyone have a slick way of combining the creation of the SConscript
while also building? I do not have a "root"(at the SConstruct level)
SConscript file - which may the be the answer - but thought I would ask
smarter people than me before I spent the time.

Post by Gary Granger

Post by Spencer Yost
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

Spencer,

I'm sure it could be done, but, it would like to be more complicated than
doing some simpler like:
Basically create your own sconsign file for these build file -> SConscript
generation
Load file
Compare timestamp and MD5 of build file, if hasn't changed since last build
do nothing
If it has, regenerate it's corresponding SConscript.

Do all this in plain python at the top of your SConstruct (possibly logic
in your site_init).
Then process SConscripts thereafter like any other build.

Next level of complication would be do as I suggested in the other email
thread, parse your build script, make the appropriate SCons calls for each
build script.
And don't generate any SConscripts at all.

When I think about these things I try not to think "What is the most
complicated way to do something", but rather, what's the simplest.
Going into the guts of SCons to do what is basically the same as either
above approach would be the most complicated solution by far.
Both in terms of cognitive load to understand all the moving pieces, and in
terms of getting and proving that it's correct (aka tests).

Einstein: “Everything should be as simple as it can be, but not simpler”

That's my two cents.

Hope that helps.
-Bill

Post by Spencer Yost
I know this has been covered before, and I apologize for plowing old
I create my SConscript files on the fly. I have my own system for
determining whether or not it needs to be built for a particular module in
the tree, then I build it if needed and then make the SConscript() call.
this works great. So I decided I want to create a builder for this
SConscript creation - just because that seemed to be the "right" way to do
it. But if I use a custom builder it will not build the software if it
determines the SConscript file needs to be built. It just creates the
SConscript file. If it doesn't have to create/recreate, it will build the
software.
I have become experienced enough with Scons now to realize why it is doing
this. But there has to be a way to get it to build the SConscript file
using a builder and build the software without running a second time. Does
anyone have a slick way of combining the creation of the SConscript while
also building? I do not have a "root"(at the SConstruct level) SConscript
file - which may the be the answer - but thought I would ask smarter
people than me before I spent the time.
Thanks in advance!
Spencer Yost
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users