corrupted objects are not rebuilt as one might expect ????

Steven Knight

2003-05-31 10:45:21 UTC

Hi Steve--

Post by Steve Allen
I saw this problem in cons 2.3 and it seems to have migrated to Scons.
It's a nit but it's pretty critical to me, since I'm hoping to extend
SCons to hand verilog and HW development environments (new file types,
lots of "strange" and unique rules, etc)....

Note that there's already a TargetSignatures('content') call that cuts
down on unnecessary downstream builds if an intermediate file is rebuilt
the same as it was last time. The corresponding feature in Cons classic
was put in largely at the request of a developer trying to eliminate
expensive hardware development steps (at Intel, IIRC).

Post by Steve Allen
For what it's worth, the same thing happens with
env.Program( "hello", "hello.cpp" );
If you corrupt hello.o or hello
This make scons unusable for me...

Curiosity question: What are the development environment issues make
corruption of files likely enough that this is a hot button?

(I'm curious because of my background with hardware developers; my day
job(s) for the past ten years have been with equipment manufacturers
with a lot of FPGA and ASIC development. So I understand that it
takes a long time to place-and-route and sim Verilog and VHDL.

What's surprising to me is that hardware development processes and
engineers--or at least those I've worked with--which are justifiably
obsessive about the physical aspects of their designs, tend to be a
*lot* more cavalier about the sort of software engineering discipline
that I would think Verilog and VHDL development would demand. There's
always a lot of grabbing files from other peoples' home directories and
the like.)

Post by Steve Allen
I'll take a stab at debugging it, but I think the basic problem is that
the signature of the object files are no longer considered for all
targets/objects. I.E. this is a design feature, not a, er, bug. My
basic understand was that a target was rebuilt at least if the signature
of the source, it's build command changed, but at one point it also got
rebuilt if the signature of the target changed also. I.E. The
signature was assumed to include the targets being built.

Well, if you mean that the target signature would include the contents
of the target, that would lead to a chicken-and-egg situation with
generating the signature in the first place.

The underlying issue here (as I see it) is that SCons believes the
signature info in the .sconsign file is accurate, and doesn't have a
way to validate that the target is unchanged since the .sconsign file
was written. We're already fighting an ongoing battle with keeping
performance acceptable as we add features, so opening up every target
file to MD5 its contents isn't attractive, at least not by default.

What I'd be inclined to support is an option that would let you specify
that you want to take the extra CPU time to validate the target files
aren't corrupt:

ValidateTargets(1)

Or, since you could conceivably validate in multiple ways, maybe
something like:

# Cheap validation, make sure the mtime hasn't changed.
ValidateTargets('timestamp')

# Expensive validation, make sure the contents haven't changed.
ValidateTargets('contents')

Comments?

Post by Steve Allen
It seems pretty broken if scons can't even detect a file change on a
object. What if a object is generated by an external program?

Substantive question: Do you have another build tool in mind that
already detects this and Does the Right Thing? Make certainly doesn't.
The only one that springs readily to mind for me is clearmake, which
could make use of the funky ClearCase file system to detect a changed
target file efficiently. (A concrete model of another tool that does
what you want is always helpful in figuring out what to make SCons do.)

Post by Steve Allen
In any case, you get the idea. I'm pretty excited about scons, so I
hope I can get this figured out....

I do, too. This would be a valuable addition. Thanks!

--SK

-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5