[Scons-users] env.Substfile() on files encoded with non UTF-8 (UTF-16).

[Dmitry Igrishin]

Hello all,

We need to process files encoded with CP1251. But Substfile() read/write by
using UTF-8 (UTF-16). :-(
What is the best workaround here?

Thanks!

[Bill Deegan]

Which version of SCons?
Which version of Python?
Which Platform? (win,mac,linux,etc)

Post by Dmitry Igrishin
Hello all,
We need to process files encoded with CP1251. But Substfile() read/write
by using UTF-8 (UTF-16). :-(
What is the best workaround here?
Thanks!
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dmitry Igrishin]

Post by Bill Deegan
Which version of SCons?
Which version of Python?
Which Platform? (win,mac,linux,etc)

SCons 3.0.0, Python 2.7.14, Windows 10 x64.

[Bill Deegan]

Do you need this for all files, or just some?

Post by Dmitry Igrishin

Post by Bill Deegan
Which version of SCons?
Which version of Python?
Which Platform? (win,mac,linux,etc)

SCons 3.0.0, Python 2.7.14, Windows 10 x64.
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dmitry Igrishin]

Post by Bill Deegan
Do you need this for all files, or just some?

Well, almost all our files are encoded with CP1251. But some files in UTF-8.
As for now all files we need to process are in CP1251. Not sure about other
parts of the project(s) (it is huge!).

[Bill Deegan]

Does this occur for SCons 2.5.1?

Post by Dmitry Igrishin

Post by Bill Deegan
Do you need this for all files, or just some?

Well, almost all our files are encoded with CP1251. But some files in UTF-8.
As for now all files we need to process are in CP1251. Not sure about
other parts of the project(s) (it is huge!).
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dmitry Igrishin]

Post by Bill Deegan
Does this occur for SCons 2.5.1?

Not tested (yet). We are moved from 2.3.6 to 3.0.0 because we are moving to
Visual Studio 2017 compiler which is not detectable by 2.3.6 (at least
here).
I guess that VS 2017 is not detectable by 2.5.1 also?..

[Mats Wichmann]

Post by Dmitry Igrishin

Post by Bill Deegan
Does this occur for SCons 2.5.1?

Not tested (yet). We are moved from 2.3.6 to 3.0.0 because we are moving to
Visual Studio 2017 compiler which is not detectable by 2.3.6 (at least
here).
I guess that VS 2017 is not detectable by 2.5.1 also?..

right, VS 2017 needs scons 3.

[Bill Deegan]

I don't have a clean fix for you, but you might try changing
get_text_contents() in SCons/Node/FS.py to be the same as get_contents()
(Comment out current get_text_contents) in class File(..), and then
get_text_contents=get_contents

See if that solves the problem for u?
-Bill

Post by Mats Wichmann

Post by Dmitry Igrishin

Post by Bill Deegan
Does this occur for SCons 2.5.1?

Not tested (yet). We are moved from 2.3.6 to 3.0.0 because we are moving

to

Post by Dmitry Igrishin
Visual Studio 2017 compiler which is not detectable by 2.3.6 (at least
here).
I guess that VS 2017 is not detectable by 2.5.1 also?..

right, VS 2017 needs scons 3.
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dmitry Igrishin]

Post by Bill Deegan
I don't have a clean fix for you, but you might try changing
get_text_contents() in SCons/Node/FS.py to be the same as get_contents()
(Comment out current get_text_contents) in class File(..), and then
get_text_contents=get_contents
See if that solves the problem for u?

Changing sources of the SCons is undesirable for us. What if monkeypatch
SCons.Node.FS.File in the custom Tool definition like that:

import sys
import re
from SCons.Builder import Builder
from SCons.Environment import *

SCons.Node.FS.File.get_text_contents = SCons.Node.FS.File.get_contents

# ...

Is it safe to monkeypatch SCons like above?

PS. I've tested it and it seems to works.

[Bill Deegan]

Yes. That should be fine.

Please file a bug in scons.tigris.org
We'll have to resolve this.

Post by Dmitry Igrishin

Post by Bill Deegan
I don't have a clean fix for you, but you might try changing
get_text_contents() in SCons/Node/FS.py to be the same as get_contents()
(Comment out current get_text_contents) in class File(..), and then
get_text_contents=get_contents
See if that solves the problem for u?

Changing sources of the SCons is undesirable for us. What if monkeypatch
import sys
import re
from SCons.Builder import Builder
from SCons.Environment import *
SCons.Node.FS.File.get_text_contents = SCons.Node.FS.File.get_contents
# ...
Is it safe to monkeypatch SCons like above?
PS. I've tested it and it seems to works.

[Tim Jenness]

You might have to add an argument somewhere to tell SCons what the encoding is in files it is parsing.

Post by Bill Deegan
Yes. That should be fine.
Please file a bug in scons.tigris.org <http://scons.tigris.org/>
We'll have to resolve this.
I don't have a clean fix for you, but you might try changing get_text_contents() in SCons/Node/FS.py to be the same as get_contents()
(Comment out current get_text_contents) in class File(..), and then get_text_contents=get_contents
See if that solves the problem for u?
import sys
import re
from SCons.Builder import Builder
from SCons.Environment import *
SCons.Node.FS.File.get_text_contents = SCons.Node.FS.File.get_contents
# ...
Is it safe to monkeypatch SCons like above?
PS. I've tested it and it seems to works.
_______________________________________________
Scons-users mailing list
https://pairlist4.pair.net/mailman/listinfo/scons-users

[Dmitry Igrishin]

Post by Bill Deegan
Yes. That should be fine.
Please file a bug in scons.tigris.org
We'll have to resolve this.

Okay. Thank you!