[Scons-users] Building object files in multiple environments

[Dan Čermák]

Hi Folks,

I am currently using SCons for a big mono-repo C++ Firmware, where I
have lots of common source files and then a directory where the actual
firmware for each device is (this is just a .cpp file with device
specific configurations and the appropriate high-level logic).

For illustration purposes, the directory structure looks something like
this:
.
├── devices
│   ├── Device_A
│   │   ├── main.cpp
│   │   ├── SConscript
│   │   ├── uart.cpp
│   │   └── uart.hpp
│   ├── Device_B
│   │   ├── main.cpp
│   │   └── SConscript
│   └── SConscript
├── spi
│   └── config.cpp
├── uart
│   └── uart.cpp
├── util
│   └── endian.c
└── SConstruct

The idea behind this is that everything outside of 'devices' is
considered as common files and build into object files. Every
subdirectory of devices should have a SConscript that creates a single
binary file (this is the firmware for the specific device) that is
linked with the common object files.

I have achieved this by creating object files for every cpp file not in
devices/ and passing them to the SConscript in devices/. However, there
is a catch: I would like to build everything under devices with an
additional include path (the top level directory of the project) which
should not be propagated to the common object files. I therefore tried
creating a clone of the environment for each device in
devices/SConscript and then doing the following in
devices/Device_A/SConscript:

Import('env_clone', 'obj')

local_obj = env_clone.Object(Glob('*.cpp'))
prog = env_clone.Program('Device_A_bin', local_obj + obj)

Return('prog')

where env_clone is the cloned environment with the additional include
path and obj the list of object files that are common to all
devices (which have been created in the environment env from which
env_clone was cloned). This however causes SCons to complain, that there
are object files in multiple environments with the same build command.


My guess is, that my solution of cloning environments is not the correct
way. Does someone have an idea how to achieve this with SCons?


Thanks in advance,

Dan

[Hua Yanghao]

I think it is much easier to introduce something called "configuration".
Take a look at the Linux kernel or u-boot Kconfig system, where
everything is considered equally as source code and then the
configuration file selects which pieces you want to compile and link
together.

My current project folder structure is something like this:
***@grass:~/git/usw $ tree -L 1
.
├── arch
├── boards
├── build_list.txt
├── common
├── configs
├── doc
├── drivers
├── external
├── include
├── lib
├── Makefile
├── output
├── README.md
├── SConstruct
├── site_scons
├── soc
├── test
├── TODO.md
└── tools

14 directories, 5 files
***@grass:~/git/usw $

where in configs/ you have all the configuration file written in
python, for example configs/qemu_arm_vexpress_bios.py: 1
#!/usr/bin/env python
2 '''
3 To execute:
4 export QEMU_AUDIO_DRV="none"
5 qemu-system-arm -M vexpress-a9 -nographic -kernel
output/armv7_gcc_full/usw.elf
6 '''
7 from common import ConfigMeta
8 from compiler import gcc_arm_none_eabi
9
10 class Config(object):
11 __metaclass__ = ConfigMeta
12 COMPILER = gcc_arm_none_eabi.Compiler()
13
14 CONFIG_ARCH = "armv7"
15 CONFIG_NO_STACK = 1
16 CONFIG_BIOS_START = 1
17 CONFIG_BOOT_ENTRY = 0x60000200
18 CONFIG_LINK_SCRIPT = "arch/armv7/link.ld"
19 CONFIG_TEXT_START = 0x00000000
20 CONFIG_HEAP_SIZE = 0x100000
21 CONFIG_ARM_BASE = 0x1e000000
22 CONFIG_UART_BASE = 0x10009000
23 CONFIG_PRINT_COLOR = 1
24
25 CONFIG_IBI_HEADER = 1
26 CONFIG_IBI_SIZELIMIT = 0x64000
27 # SMP
28 CONFIG_CPU_NUM = 4
29 CONFIG_IRQ_STACK_SIZE = 0x10000
30 CONFIG_FIQ_STACK_SIZE = 0x10000
31
32 # Module List
33 MODULE_LIST = [
34 "arch",
35 "arch/armv7",
36 ]

This is one of my simplest config file, where it selects which module
(e.g. each folder that have a SConscript is considered a module) is
specified in the MODULE_LIST and then in SConstruct the SConscript
file is automatically imported.
All build that is corresponding to a particular config file has a
dedicated output folder:
***@grass:~/git/usw $ ll output/
total 28
drwxr-xr-x 9 hua hua 4096 Nov 8 10:30 linux64_full
drwxr-xr-x 10 hua hua 4096 Nov 8 10:29 qemu_arm_vexpress
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 qemu_arm_vexpress_bios
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_lmu
drwxr-xr-x 8 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_yyy
***@grass:~/git/usw $

So even for a single device, you could have have different
configuration, e.g. one for boot loader/bios, one for verification SW,
one for production SW etc.

The best part I love SCons is that in the output folder there is a
copy of the file that is actually being selected/used from the
configuration system so you know exactly what is compiled.

For each individual module the SConscript is extremely simple and
straightforward:

***@grass:~/git/usw $ cat boards/vexpress/SConscript
name = "usw_lib"

Import('cs')
Import('usw_files')

obj_files = [
"board.c",
"pl011_device.c",
"test_device.c",
"pipe_device.c",
]

lib_files = [
]

if hasattr(cs, "CONFIG_LOAD_BINARY"):
obj_files.append(("binary.c", ["cmd.txt", "page_table.bin"]))

ret = usw_files(name, lib_files, obj_files)

Return('ret')
***@grass:~/git/usw $

There you just provide a obj_files / lib_files and you can even
specify dependency files (where you don't really want a Scons parser),
and even specify customized compilation flags for individual files
(Kconfig/Kbuild can also do this, but not the explicit dependency
specification).

I am trying to make this framework (basically Kconfig/Kbuild reduced
feature set implemented using SCons) open source but company process
is lengthy ...
Or maybe it even makes sense to make this kind of feature as part of
SCons itself?

I hope this helps a little bit.

Best Regards,
Hua Yanghao

On Wed, Nov 8, 2017 at 10:48 AM, Dan Čermák

Post by Dan Čermák
Hi Folks,
I am currently using SCons for a big mono-repo C++ Firmware, where I
have lots of common source files and then a directory where the actual
firmware for each device is (this is just a .cpp file with device
specific configurations and the appropriate high-level logic).
For illustration purposes, the directory structure looks something like
.
├── devices
│ ├── Device_A
│ │ ├── main.cpp
│ │ ├── SConscript
│ │ ├── uart.cpp
│ │ └── uart.hpp
│ ├── Device_B
│ │ ├── main.cpp
│ │ └── SConscript
│ └── SConscript
├── spi
│ └── config.cpp
├── uart
│ └── uart.cpp
├── util
│ └── endian.c
└── SConstruct
The idea behind this is that everything outside of 'devices' is
considered as common files and build into object files. Every
subdirectory of devices should have a SConscript that creates a single
binary file (this is the firmware for the specific device) that is
linked with the common object files.
I have achieved this by creating object files for every cpp file not in
devices/ and passing them to the SConscript in devices/. However, there
is a catch: I would like to build everything under devices with an
additional include path (the top level directory of the project) which
should not be propagated to the common object files. I therefore tried
creating a clone of the environment for each device in
devices/SConscript and then doing the following in
Import('env_clone', 'obj')
local_obj = env_clone.Object(Glob('*.cpp'))
prog = env_clone.Program('Device_A_bin', local_obj + obj)
Return('prog')
where env_clone is the cloned environment with the additional include
path and obj the list of object files that are common to all
devices (which have been created in the environment env from which
env_clone was cloned). This however causes SCons to complain, that there
are object files in multiple environments with the same build command.
My guess is, that my solution of cloning environments is not the correct
way. Does someone have an idea how to achieve this with SCons?
Thanks in advance,
Dan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dan Čermák]

Thanks a lot for your extensive answer!

Do I understand it correctly that for each "device"/"project" you infer
all modules from the config file and then build them for that specific
project?

I actually wanted it kind of the other way around: all modules are build
once and then linked for each project, but the project specific stuff
should be build with an additional include path.

I must admit that I like your approach very much and I think I could use
something like that, too. Could you give me maybe a hint how your
configuration system roughly works (only if you can do that without
getting into trouble with your employer)?


Cheers,

Dan

Post by Hua Yanghao
I think it is much easier to introduce something called "configuration".
Take a look at the Linux kernel or u-boot Kconfig system, where
everything is considered equally as source code and then the
configuration file selects which pieces you want to compile and link
together.
.
├── arch
├── boards
├── build_list.txt
├── common
├── configs
├── doc
├── drivers
├── external
├── include
├── lib
├── Makefile
├── output
├── README.md
├── SConstruct
├── site_scons
├── soc
├── test
├── TODO.md
└── tools
14 directories, 5 files
where in configs/ you have all the configuration file written in
python, for example configs/qemu_arm_vexpress_bios.py: 1
#!/usr/bin/env python
2 '''
4 export QEMU_AUDIO_DRV="none"
5 qemu-system-arm -M vexpress-a9 -nographic -kernel
output/armv7_gcc_full/usw.elf
6 '''
7 from common import ConfigMeta
8 from compiler import gcc_arm_none_eabi
9
11 __metaclass__ = ConfigMeta
12 COMPILER = gcc_arm_none_eabi.Compiler()
13
14 CONFIG_ARCH = "armv7"
15 CONFIG_NO_STACK = 1
16 CONFIG_BIOS_START = 1
17 CONFIG_BOOT_ENTRY = 0x60000200
18 CONFIG_LINK_SCRIPT = "arch/armv7/link.ld"
19 CONFIG_TEXT_START = 0x00000000
20 CONFIG_HEAP_SIZE = 0x100000
21 CONFIG_ARM_BASE = 0x1e000000
22 CONFIG_UART_BASE = 0x10009000
23 CONFIG_PRINT_COLOR = 1
24
25 CONFIG_IBI_HEADER = 1
26 CONFIG_IBI_SIZELIMIT = 0x64000
27 # SMP
28 CONFIG_CPU_NUM = 4
29 CONFIG_IRQ_STACK_SIZE = 0x10000
30 CONFIG_FIQ_STACK_SIZE = 0x10000
31
32 # Module List
33 MODULE_LIST = [
34 "arch",
35 "arch/armv7",
36 ]
This is one of my simplest config file, where it selects which module
(e.g. each folder that have a SConscript is considered a module) is
specified in the MODULE_LIST and then in SConstruct the SConscript
file is automatically imported.
All build that is corresponding to a particular config file has a
total 28
drwxr-xr-x 9 hua hua 4096 Nov 8 10:30 linux64_full
drwxr-xr-x 10 hua hua 4096 Nov 8 10:29 qemu_arm_vexpress
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 qemu_arm_vexpress_bios
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_lmu
drwxr-xr-x 8 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_yyy
So even for a single device, you could have have different
configuration, e.g. one for boot loader/bios, one for verification SW,
one for production SW etc.
The best part I love SCons is that in the output folder there is a
copy of the file that is actually being selected/used from the
configuration system so you know exactly what is compiled.
For each individual module the SConscript is extremely simple and
name = "usw_lib"
Import('cs')
Import('usw_files')
obj_files = [
"board.c",
"pl011_device.c",
"test_device.c",
"pipe_device.c",
]
lib_files = [
]
obj_files.append(("binary.c", ["cmd.txt", "page_table.bin"]))
ret = usw_files(name, lib_files, obj_files)
Return('ret')
There you just provide a obj_files / lib_files and you can even
specify dependency files (where you don't really want a Scons parser),
and even specify customized compilation flags for individual files
(Kconfig/Kbuild can also do this, but not the explicit dependency
specification).
I am trying to make this framework (basically Kconfig/Kbuild reduced
feature set implemented using SCons) open source but company process
is lengthy ...
Or maybe it even makes sense to make this kind of feature as part of
SCons itself?
I hope this helps a little bit.
Best Regards,
Hua Yanghao
On Wed, Nov 8, 2017 at 10:48 AM, Dan Čermák

Post by Dan Čermák
Hi Folks,
I am currently using SCons for a big mono-repo C++ Firmware, where I
have lots of common source files and then a directory where the actual
firmware for each device is (this is just a .cpp file with device
specific configurations and the appropriate high-level logic).
For illustration purposes, the directory structure looks something like
.
├── devices
│ ├── Device_A
│ │ ├── main.cpp
│ │ ├── SConscript
│ │ ├── uart.cpp
│ │ └── uart.hpp
│ ├── Device_B
│ │ ├── main.cpp
│ │ └── SConscript
│ └── SConscript
├── spi
│ └── config.cpp
├── uart
│ └── uart.cpp
├── util
│ └── endian.c
└── SConstruct
The idea behind this is that everything outside of 'devices' is
considered as common files and build into object files. Every
subdirectory of devices should have a SConscript that creates a single
binary file (this is the firmware for the specific device) that is
linked with the common object files.
I have achieved this by creating object files for every cpp file not in
devices/ and passing them to the SConscript in devices/. However, there
is a catch: I would like to build everything under devices with an
additional include path (the top level directory of the project) which
should not be propagated to the common object files. I therefore tried
creating a clone of the environment for each device in
devices/SConscript and then doing the following in
Import('env_clone', 'obj')
local_obj = env_clone.Object(Glob('*.cpp'))
prog = env_clone.Program('Device_A_bin', local_obj + obj)
Return('prog')
where env_clone is the cloned environment with the additional include
path and obj the list of object files that are common to all
devices (which have been created in the environment env from which
env_clone was cloned). This however causes SCons to complain, that there
are object files in multiple environments with the same build command.
My guess is, that my solution of cloning environments is not the correct
way. Does someone have an idea how to achieve this with SCons?
Thanks in advance,
Dan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Hua Yanghao]

Hi Dan,
The problem with building everything into a library and only linking
in the project specific part is that you assume all your project are
using the same CPU architecture/same compiler.
If you want to cover all different types of CPUs/Project in a single
repo ... building a separate library does not make too much sense.

FYI the SoC I am working on have 4 different CPU architecture on a
single project.

Regarding how it works it is really designing quite some python
modules and use scons as a library to handle the dependency and job
dispatching.
It is currently less than 4K lines of code but it involves quite some
design. However, everything boils down to import module SConscript and
adding necessary include path for that module.

Basically this is how it works:
* SConstruct is the entry to read in the config file and import
necessary modules
* Each module at is level only specifies the source file, and return a
list of source file to the toplevel SConstruct file.
* Top level SConstruct file will then build each module source file
into object files as well as a library (lib_files/obj_files).
* SConstruct then takes all the library and object files and create
the final binary (using variant_dir for SConscript in order NOT to
build in place).
* The configuration file can be used as: 1. configuration file (of
course) 2. trigger the build (it invokes scons) 3. execute the target
(run it in emulator or real board).

Best Regards,
Yanghao

On Wed, Nov 8, 2017 at 12:38 PM, Dan Čermák

Post by Dan Čermák
Thanks a lot for your extensive answer!
Do I understand it correctly that for each "device"/"project" you infer
all modules from the config file and then build them for that specific
project?
I actually wanted it kind of the other way around: all modules are build
once and then linked for each project, but the project specific stuff
should be build with an additional include path.
I must admit that I like your approach very much and I think I could use
something like that, too. Could you give me maybe a hint how your
configuration system roughly works (only if you can do that without
getting into trouble with your employer)?
Cheers,
Dan

Post by Hua Yanghao
I think it is much easier to introduce something called "configuration".
Take a look at the Linux kernel or u-boot Kconfig system, where
everything is considered equally as source code and then the
configuration file selects which pieces you want to compile and link
together.
.
├── arch
├── boards
├── build_list.txt
├── common
├── configs
├── doc
├── drivers
├── external
├── include
├── lib
├── Makefile
├── output
├── README.md
├── SConstruct
├── site_scons
├── soc
├── test
├── TODO.md
└── tools
14 directories, 5 files
where in configs/ you have all the configuration file written in
python, for example configs/qemu_arm_vexpress_bios.py: 1
#!/usr/bin/env python
2 '''
4 export QEMU_AUDIO_DRV="none"
5 qemu-system-arm -M vexpress-a9 -nographic -kernel
output/armv7_gcc_full/usw.elf
6 '''
7 from common import ConfigMeta
8 from compiler import gcc_arm_none_eabi
9
11 __metaclass__ = ConfigMeta
12 COMPILER = gcc_arm_none_eabi.Compiler()
13
14 CONFIG_ARCH = "armv7"
15 CONFIG_NO_STACK = 1
16 CONFIG_BIOS_START = 1
17 CONFIG_BOOT_ENTRY = 0x60000200
18 CONFIG_LINK_SCRIPT = "arch/armv7/link.ld"
19 CONFIG_TEXT_START = 0x00000000
20 CONFIG_HEAP_SIZE = 0x100000
21 CONFIG_ARM_BASE = 0x1e000000
22 CONFIG_UART_BASE = 0x10009000
23 CONFIG_PRINT_COLOR = 1
24
25 CONFIG_IBI_HEADER = 1
26 CONFIG_IBI_SIZELIMIT = 0x64000
27 # SMP
28 CONFIG_CPU_NUM = 4
29 CONFIG_IRQ_STACK_SIZE = 0x10000
30 CONFIG_FIQ_STACK_SIZE = 0x10000
31
32 # Module List
33 MODULE_LIST = [
34 "arch",
35 "arch/armv7",
36 ]
This is one of my simplest config file, where it selects which module
(e.g. each folder that have a SConscript is considered a module) is
specified in the MODULE_LIST and then in SConstruct the SConscript
file is automatically imported.
All build that is corresponding to a particular config file has a
total 28
drwxr-xr-x 9 hua hua 4096 Nov 8 10:30 linux64_full
drwxr-xr-x 10 hua hua 4096 Nov 8 10:29 qemu_arm_vexpress
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 qemu_arm_vexpress_bios
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_lmu
drwxr-xr-x 8 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_yyy
So even for a single device, you could have have different
configuration, e.g. one for boot loader/bios, one for verification SW,
one for production SW etc.
The best part I love SCons is that in the output folder there is a
copy of the file that is actually being selected/used from the
configuration system so you know exactly what is compiled.
For each individual module the SConscript is extremely simple and
name = "usw_lib"
Import('cs')
Import('usw_files')
obj_files = [
"board.c",
"pl011_device.c",
"test_device.c",
"pipe_device.c",
]
lib_files = [
]
obj_files.append(("binary.c", ["cmd.txt", "page_table.bin"]))
ret = usw_files(name, lib_files, obj_files)
Return('ret')
There you just provide a obj_files / lib_files and you can even
specify dependency files (where you don't really want a Scons parser),
and even specify customized compilation flags for individual files
(Kconfig/Kbuild can also do this, but not the explicit dependency
specification).
I am trying to make this framework (basically Kconfig/Kbuild reduced
feature set implemented using SCons) open source but company process
is lengthy ...
Or maybe it even makes sense to make this kind of feature as part of
SCons itself?
I hope this helps a little bit.
Best Regards,
Hua Yanghao
On Wed, Nov 8, 2017 at 10:48 AM, Dan Čermák

Post by Dan Čermák
Hi Folks,
I am currently using SCons for a big mono-repo C++ Firmware, where I
have lots of common source files and then a directory where the actual
firmware for each device is (this is just a .cpp file with device
specific configurations and the appropriate high-level logic).
For illustration purposes, the directory structure looks something like
.
├── devices
│ ├── Device_A
│ │ ├── main.cpp
│ │ ├── SConscript
│ │ ├── uart.cpp
│ │ └── uart.hpp
│ ├── Device_B
│ │ ├── main.cpp
│ │ └── SConscript
│ └── SConscript
├── spi
│ └── config.cpp
├── uart
│ └── uart.cpp
├── util
│ └── endian.c
└── SConstruct
The idea behind this is that everything outside of 'devices' is
considered as common files and build into object files. Every
subdirectory of devices should have a SConscript that creates a single
binary file (this is the firmware for the specific device) that is
linked with the common object files.
I have achieved this by creating object files for every cpp file not in
devices/ and passing them to the SConscript in devices/. However, there
is a catch: I would like to build everything under devices with an
additional include path (the top level directory of the project) which
should not be propagated to the common object files. I therefore tried
creating a clone of the environment for each device in
devices/SConscript and then doing the following in
Import('env_clone', 'obj')
local_obj = env_clone.Object(Glob('*.cpp'))
prog = env_clone.Program('Device_A_bin', local_obj + obj)
Return('prog')
where env_clone is the cloned environment with the additional include
path and obj the list of object files that are common to all
devices (which have been created in the environment env from which
env_clone was cloned). This however causes SCons to complain, that there
are object files in multiple environments with the same build command.
My guess is, that my solution of cloning environments is not the correct
way. Does someone have an idea how to achieve this with SCons?
Thanks in advance,
Dan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dan Čermák]

Hi Yanghao,

thank you for the elaborate answer. I must admit I begin to like the
idea of having a config file for each device and letting scons piece
everything together. I'll give this a try but I don't think that I'll
create a really generic tool, since my use case is a lot narrower than
yours (I am only building for one platform and I'd like the config file
to mainly handle the proper initialization and configuration of the
CPU).


Cheers,

Dan

Post by Hua Yanghao
Hi Dan,
The problem with building everything into a library and only linking
in the project specific part is that you assume all your project are
using the same CPU architecture/same compiler.
If you want to cover all different types of CPUs/Project in a single
repo ... building a separate library does not make too much sense.
FYI the SoC I am working on have 4 different CPU architecture on a
single project.
Regarding how it works it is really designing quite some python
modules and use scons as a library to handle the dependency and job
dispatching.
It is currently less than 4K lines of code but it involves quite some
design. However, everything boils down to import module SConscript and
adding necessary include path for that module.
* SConstruct is the entry to read in the config file and import
necessary modules
* Each module at is level only specifies the source file, and return a
list of source file to the toplevel SConstruct file.
* Top level SConstruct file will then build each module source file
into object files as well as a library (lib_files/obj_files).
* SConstruct then takes all the library and object files and create
the final binary (using variant_dir for SConscript in order NOT to
build in place).
* The configuration file can be used as: 1. configuration file (of
course) 2. trigger the build (it invokes scons) 3. execute the target
(run it in emulator or real board).
Best Regards,
Yanghao
On Wed, Nov 8, 2017 at 12:38 PM, Dan Čermák

Post by Dan Čermák
Thanks a lot for your extensive answer!
Do I understand it correctly that for each "device"/"project" you infer
all modules from the config file and then build them for that specific
project?
I actually wanted it kind of the other way around: all modules are build
once and then linked for each project, but the project specific stuff
should be build with an additional include path.
I must admit that I like your approach very much and I think I could use
something like that, too. Could you give me maybe a hint how your
configuration system roughly works (only if you can do that without
getting into trouble with your employer)?
Cheers,
Dan

Post by Hua Yanghao
I think it is much easier to introduce something called "configuration".
Take a look at the Linux kernel or u-boot Kconfig system, where
everything is considered equally as source code and then the
configuration file selects which pieces you want to compile and link
together.
.
├── arch
├── boards
├── build_list.txt
├── common
├── configs
├── doc
├── drivers
├── external
├── include
├── lib
├── Makefile
├── output
├── README.md
├── SConstruct
├── site_scons
├── soc
├── test
├── TODO.md
└── tools
14 directories, 5 files
where in configs/ you have all the configuration file written in
python, for example configs/qemu_arm_vexpress_bios.py: 1
#!/usr/bin/env python
2 '''
4 export QEMU_AUDIO_DRV="none"
5 qemu-system-arm -M vexpress-a9 -nographic -kernel
output/armv7_gcc_full/usw.elf
6 '''
7 from common import ConfigMeta
8 from compiler import gcc_arm_none_eabi
9
11 __metaclass__ = ConfigMeta
12 COMPILER = gcc_arm_none_eabi.Compiler()
13
14 CONFIG_ARCH = "armv7"
15 CONFIG_NO_STACK = 1
16 CONFIG_BIOS_START = 1
17 CONFIG_BOOT_ENTRY = 0x60000200
18 CONFIG_LINK_SCRIPT = "arch/armv7/link.ld"
19 CONFIG_TEXT_START = 0x00000000
20 CONFIG_HEAP_SIZE = 0x100000
21 CONFIG_ARM_BASE = 0x1e000000
22 CONFIG_UART_BASE = 0x10009000
23 CONFIG_PRINT_COLOR = 1
24
25 CONFIG_IBI_HEADER = 1
26 CONFIG_IBI_SIZELIMIT = 0x64000
27 # SMP
28 CONFIG_CPU_NUM = 4
29 CONFIG_IRQ_STACK_SIZE = 0x10000
30 CONFIG_FIQ_STACK_SIZE = 0x10000
31
32 # Module List
33 MODULE_LIST = [
34 "arch",
35 "arch/armv7",
36 ]
This is one of my simplest config file, where it selects which module
(e.g. each folder that have a SConscript is considered a module) is
specified in the MODULE_LIST and then in SConstruct the SConscript
file is automatically imported.
All build that is corresponding to a particular config file has a
total 28
drwxr-xr-x 9 hua hua 4096 Nov 8 10:30 linux64_full
drwxr-xr-x 10 hua hua 4096 Nov 8 10:29 qemu_arm_vexpress
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 qemu_arm_vexpress_bios
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0
drwxr-xr-x 11 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx
drwxr-xr-x 4 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_lmu
drwxr-xr-x 8 hua hua 4096 Nov 8 10:30 xmm7xx0_xxx_yyy
So even for a single device, you could have have different
configuration, e.g. one for boot loader/bios, one for verification SW,
one for production SW etc.
The best part I love SCons is that in the output folder there is a
copy of the file that is actually being selected/used from the
configuration system so you know exactly what is compiled.
For each individual module the SConscript is extremely simple and
name = "usw_lib"
Import('cs')
Import('usw_files')
obj_files = [
"board.c",
"pl011_device.c",
"test_device.c",
"pipe_device.c",
]
lib_files = [
]
obj_files.append(("binary.c", ["cmd.txt", "page_table.bin"]))
ret = usw_files(name, lib_files, obj_files)
Return('ret')
There you just provide a obj_files / lib_files and you can even
specify dependency files (where you don't really want a Scons parser),
and even specify customized compilation flags for individual files
(Kconfig/Kbuild can also do this, but not the explicit dependency
specification).
I am trying to make this framework (basically Kconfig/Kbuild reduced
feature set implemented using SCons) open source but company process
is lengthy ...
Or maybe it even makes sense to make this kind of feature as part of
SCons itself?
I hope this helps a little bit.
Best Regards,
Hua Yanghao
On Wed, Nov 8, 2017 at 10:48 AM, Dan Čermák

Post by Dan Čermák
Hi Folks,
I am currently using SCons for a big mono-repo C++ Firmware, where I
have lots of common source files and then a directory where the actual
firmware for each device is (this is just a .cpp file with device
specific configurations and the appropriate high-level logic).
For illustration purposes, the directory structure looks something like
.
├── devices
│ ├── Device_A
│ │ ├── main.cpp
│ │ ├── SConscript
│ │ ├── uart.cpp
│ │ └── uart.hpp
│ ├── Device_B
│ │ ├── main.cpp
│ │ └── SConscript
│ └── SConscript
├── spi
│ └── config.cpp
├── uart
│ └── uart.cpp
├── util
│ └── endian.c
└── SConstruct
The idea behind this is that everything outside of 'devices' is
considered as common files and build into object files. Every
subdirectory of devices should have a SConscript that creates a single
binary file (this is the firmware for the specific device) that is
linked with the common object files.
I have achieved this by creating object files for every cpp file not in
devices/ and passing them to the SConscript in devices/. However, there
is a catch: I would like to build everything under devices with an
additional include path (the top level directory of the project) which
should not be propagated to the common object files. I therefore tried
creating a clone of the environment for each device in
devices/SConscript and then doing the following in
Import('env_clone', 'obj')
local_obj = env_clone.Object(Glob('*.cpp'))
prog = env_clone.Program('Device_A_bin', local_obj + obj)
Return('prog')
where env_clone is the cloned environment with the additional include
path and obj the list of object files that are common to all
devices (which have been created in the environment env from which
env_clone was cloned). This however causes SCons to complain, that there
are object files in multiple environments with the same build command.
My guess is, that my solution of cloning environments is not the correct
way. Does someone have an idea how to achieve this with SCons?
Thanks in advance,
Dan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Bill Deegan]

I'd suggest you us VariantDir per device.
That's likely the simplest solution.
You won't have to change much.

Post by Dan Čermák
Hi Folks,
I am currently using SCons for a big mono-repo C++ Firmware, where I
have lots of common source files and then a directory where the actual
firmware for each device is (this is just a .cpp file with device
specific configurations and the appropriate high-level logic).
For illustration purposes, the directory structure looks something like
.
├── devices
│ ├── Device_A
│ │ ├── main.cpp
│ │ ├── SConscript
│ │ ├── uart.cpp
│ │ └── uart.hpp
│ ├── Device_B
│ │ ├── main.cpp
│ │ └── SConscript
│ └── SConscript
├── spi
│ └── config.cpp
├── uart
│ └── uart.cpp
├── util
│ └── endian.c
└── SConstruct
The idea behind this is that everything outside of 'devices' is
considered as common files and build into object files. Every
subdirectory of devices should have a SConscript that creates a single
binary file (this is the firmware for the specific device) that is
linked with the common object files.
I have achieved this by creating object files for every cpp file not in
devices/ and passing them to the SConscript in devices/. However, there
is a catch: I would like to build everything under devices with an
additional include path (the top level directory of the project) which
should not be propagated to the common object files. I therefore tried
creating a clone of the environment for each device in
devices/SConscript and then doing the following in
Import('env_clone', 'obj')
local_obj = env_clone.Object(Glob('*.cpp'))
prog = env_clone.Program('Device_A_bin', local_obj + obj)
Return('prog')
where env_clone is the cloned environment with the additional include
path and obj the list of object files that are common to all
devices (which have been created in the environment env from which
env_clone was cloned). This however causes SCons to complain, that there
are object files in multiple environments with the same build command.
My guess is, that my solution of cloning environments is not the correct
way. Does someone have an idea how to achieve this with SCons?
Thanks in advance,
Dan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dan Čermák]

Thanks for the tip, I'll give that a try.

Post by Bill Deegan
I'd suggest you us VariantDir per device.
That's likely the simplest solution.
You won't have to change much.

Post by Dan Čermák
Hi Folks,
I am currently using SCons for a big mono-repo C++ Firmware, where I
have lots of common source files and then a directory where the actual
firmware for each device is (this is just a .cpp file with device
specific configurations and the appropriate high-level logic).
For illustration purposes, the directory structure looks something like
.
├── devices
│ ├── Device_A
│ │ ├── main.cpp
│ │ ├── SConscript
│ │ ├── uart.cpp
│ │ └── uart.hpp
│ ├── Device_B
│ │ ├── main.cpp
│ │ └── SConscript
│ └── SConscript
├── spi
│ └── config.cpp
├── uart
│ └── uart.cpp
├── util
│ └── endian.c
└── SConstruct
The idea behind this is that everything outside of 'devices' is
considered as common files and build into object files. Every
subdirectory of devices should have a SConscript that creates a single
binary file (this is the firmware for the specific device) that is
linked with the common object files.
I have achieved this by creating object files for every cpp file not in
devices/ and passing them to the SConscript in devices/. However, there
is a catch: I would like to build everything under devices with an
additional include path (the top level directory of the project) which
should not be propagated to the common object files. I therefore tried
creating a clone of the environment for each device in
devices/SConscript and then doing the following in
Import('env_clone', 'obj')
local_obj = env_clone.Object(Glob('*.cpp'))
prog = env_clone.Program('Device_A_bin', local_obj + obj)
Return('prog')
where env_clone is the cloned environment with the additional include
path and obj the list of object files that are common to all
devices (which have been created in the environment env from which
env_clone was cloned). This however causes SCons to complain, that there
are object files in multiple environments with the same build command.
My guess is, that my solution of cloning environments is not the correct
way. Does someone have an idea how to achieve this with SCons?
Thanks in advance,
Dan
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users