CONVMV(1)CONVMV(1)NAMEconvmv - converts filenames from one encoding to another
SYNOPSISconvmv [options] FILE(S) ... DIRECTORY(S)OPTIONS-f ENCODING
specify the current encoding of the filename(s) from which should
be converted
-t ENCODING
specify the encoding to which the filename(s) should be converted
-i interactive mode (ask y/n for each action)
-r recursively go through directories
--nfc
target files will be normalization form C for UTF-8 (Linux etc.)
--nfd
target files will be normalization form D for UTF-8 (OS X etc.).
--qfrom , --qto
be more quiet about the "from" or "to" of a rename (if it screws up
your terminal e.g.). This will in fact do nothing else than replace
any non-ASCII character (bytewise) with ? and any control character
with * on printout, this does not affect rename operation itself.
--exec command
execute the given command. You have to quote the command and #1
will be substituted by the old, #2 by the new filename. Using this
option link targets will stay untouched.
Example:
convmv-f latin1 -t utf-8 -r --exec "echo #1 should be renamed to
#2" path/to/files
--list
list all available encodings. To get support for more Chinese or
Japanese encodings install the Perl HanExtra or JIS2K Encode pack‐
ages.
--lowmem
keep memory footprint low by not creating a hash of all files. This
disables checking if symlink targets are in subtree. Symlink target
pointers will be converted regardlessly. If you convert multiple
hundredthousands or millions of files the memory usage of convmv
might grow quite high. This option would help you out in that case.
--nosmart
by default convmv will detect if a filename is already UTF8 encoded
and will skip this file if conversion from some charset to UTF8
should be performed. "--nosmart" will also force conversion to
UTF-8 for such files, which might result in "double encoded UTF-8"
(see section below).
--notest
Needed to actually rename the files. By default convmv will just
print what it wants to do.
--replace
if the file to which shall be renamed already exists, it will be
overwritten if the other file content is equal.
--unescape
this option will remove this ugly % hex sequences from filenames
and turn them into (hopefully) nicer 8-bit characters. After
--unescape you might want to do a charset conversion. This
sequences like %20 etc. are sometimes produced when downloading via
http or ftp.
--upper , --lower
turn filenames into all upper or all lower case. When the file is
not ASCII-encoded, convmv expects a charset to be entered via the
-f switch.
--dotlessi
care about the dotless i/I issue. A lowercase version of "I" will
also be dotless while an uppercase version of "i" will also be dot‐
ted. This is an issue for Turkish and Azeri.
By the way: The superscript dot of the letter i was added in the
Middle Ages to distinguish the letter (in manuscripts) from adja‐
cent vertical strokes in such letters as u, m, and n. J is a vari‐
ant form of i which emerged at this time and subsequently became a
separate letter.
--help
print a short summary of available options
DESCRIPTIONconvmv is meant to help convert a single filename, a directory tree and
the contained files or a whole filesystem into a different encoding. It
just converts the filenames, not the content of the files. A special
feature of convmv is that it also takes care of symlinks, also converts
the symlink target pointer in case the symlink target is being con‐
verted, too.
All this comes in very handy when one wants to switch over from old
8-bit locales to UTF-8 locales. It is also possible to convert directo‐
ries to UTF-8 which are already partly UTF-8 encoded. convmv is able to
detect if certain files are UTF-8 encoded and will skip them by
default. To turn this smartness off use the "--nosmart" switch.
Filesystem issues
Almost all POSIX filesystems do not care about how filenames are
encoded, here are some exceptions:
HFS+ on OS X / Darwin
Linux and (most?) other Unix-like operating systems use the so called
normalization form C (NFC) for its UTF-8 encoding by default but do not
enforce this. Darwin, the base of the Macintosh OS enforces normaliza‐
tion form D (NFD), where a few characters are encoded in a different
way. On OS X it's not possible to create NFC UTF-8 filenames because
this is prevented at filesystem layer. On HFS+ filenames are inter‐
nally stored in UTF-16 and when converted back to UTF-8, for the under‐
lying BSD system to be handable, NFD is created. If someone knows why
Apple chose to do this, please let me know. I think it was a very bad
idea and breaks many things under OS X which expect a normal POSIX con‐
forming system. Anywhere else convmv is able to convert files from NFC
to NFD or vice versa which makes interoperability with such systems a
lot easier.
JFS
If people mount JFS partitions with iocharset=utf8, there is a similar
problem, because JFS is designed to store finenames internally in
UTF-16, too; that is because Linux' JFS is really JFS2, which was a re‐
write of JFS for OS/2. JFS partitions should always be mounted with
iocharset=iso8859-1, which is also the default with recent 2.6.6 ker‐
nels. If this is not done, JFS does not behave like a POSIX filesystem
and it might happen that certain files cannot be created at all, for
example filenames in ISO-8859-1 encoding. Only when interoperation with
OS/2 is needed iocharset should be set according to your used locale
charmap.
NFS4
Despite other POSIX filesystems RFC3530 (NFS 4) mandates UTF-8 but also
says: "The nfs4_cs_prep profile does not specify a normalization form.
A later revision of this specification may specify a particular normal‐
ization form." In other words, if you want to use NFS4 you might find
the conversion and normalization features of convmv quite useful.
FAT/VFAT and NTFS
NTFS and VFAT (for long filenames) use UTF-16 internally to store file‐
names. You should not need to convert filenames if you mount one of
those filesystems. Use appropriate mount options instead!
How to undo double UTF-8 (or other) encoded filenames
Sometimes it might happen that you "double-encoded" certain filenames,
for example the file names already were UTF-8 encoded and you acci‐
dently did another conversion from some charset to UTF-8. You can sim‐
ply undo that by converting that the other way round. The from-charset
has to be UTF-8 and the to-charset has to be the from-charset you pre‐
viously accidently used. You should check to get the correct results by
doing the conversion without "--notest" before, also the "--qfrom"
option might be helpful, because the double utf-8 file names might
screw up your terminal if they are being printed - they often contain
control sequences which do funny things with your terminal window. If
you are not sure about the charset which was accidently converted from,
using "--qfrom" is a good way to fiddle out the required encoding with‐
out destroying the file names finally.
How to repair Samba files
When in the smb.conf (of Samba 2.x) there hasn't been set a correct
"character set" variable, files which are created from Win* clients are
being created in the client's codepage, e.g. cp850 for western european
languages. As a result of that the files which contain non-ASCII char‐
acters are screwed up if you "ls" them on the Unix server. If you
change the "character set" variable afterwards to iso8859-1, newly cre‐
ated files are okay, but the old files are still screwed up in the Win‐
dows encoding. In this case convmv can also be used to convert the old
Samba-shared files from cp850 to iso8859-1.
By the way: Samba 3.x finally maps to UTF-8 filenames by default, so
also when you migrate from Samba 2 to Samba 3 you might have to convert
your file names.
SEE ALSOlocale(1)utf-8(7)charsets(7)BUGS
no bugs or fleas known
AUTHOR
Bjoern JACKE
Send mail to bjoern [at] j3e.de for bug reports and suggestions.
perl v5.8.8 2006-06-25 CONVMV(1)
