一个实现 LFS(Linux From Scratch) 自动化的脚本(上)

Submitted by Dot on Fri, 12/09/2016 - 09:20

LFS ( Linux From Scratch ) , 即“从零开始构建 linux ”。具体是指,从最基本的 glibc, binutils, gcc 等开始,逐个编译 linux 运行所必须的各个软件,最终构建出一个可以启动并独立运行的 linux 。
一个完整的LFS过程至少要分为两个阶段—— chroot 前和后。本文涉及 chroot 前的内容,之后的内容将来同样会以脚本的形式发布。届时本文也会最终完善。

00 这个脚本能做什么?

基本可以完成 LFS 7.10 Chapter 6.4 即 chroot (包括)之前的所有工作,包括软件包下载、环境设定、编译等,直到 chroot 命令本身。

01 “基本”的意思是还少点什么?

  • 没有检查主机包安装情况。这通常不是问题,因为对于发行版来说,大多数包都已安装,你可能只需要装下 bison 。并且2.2节有现成的脚本

  • 没有创建用户 lfs 。lfs 用户的作用是创造一个相对干净的环境,相关设置有4项:set +humask 002 和 env 都已写入脚本,而 useradd -k /dev/null 的目的是屏蔽 ~/.bashrc 和 ~/.bash_profile ,作为 non-interactive shell 的脚本本身就无视这些。
    但是,尽管如此,在运行这个脚本前,还是应该参照 LFS 建议,进入一个隔离的环境。用 env -i sh 或 env -i bash --norc 来切换即可。正常环境中通常也能顺利完成,但潜在问题无法预知。

02 完全遵循LFS么?

基本上是的,流程及各个参数均遵循 LFS 7.10 。有些地方可能看起来和手册中有差别,实际上只是在不影响效果的前提下为代码简洁而做的调整。比如,定义变量 configure=../configure,编译 libstdc++ 前将其赋值 configure=../libstdc++-v3/configure,这就省去了为 configure 创建 case 分支。

03 怎么用?我要做什么吗?

  1. 设定环境变量 LFS
  2. 挂载 LFS 分区
  3. 参照2.2检查主机软件包
  4. 运行脚本(先查看 --help )

04 遇到的问题

  • 整个 chapter 5 都顺利编译,包括 binutils 和 gcc 的两个 pass 。但 chroot 后编译 6.7. Linux-4.7.2 API Headers 时报错,进而发现 gcc 无法运行, gcc: Command not found,而 gcc 执行文件明明就在那里;
  • 退出 choot ,在 host 上用 $LFS/tools/bin/gcc 则可以运行;
  • file /tools/bin/gcc 反馈 interpreter /lib64/ld-linux-x86-64.so.2 ,而在 pass 2 之后,binutils 和 gcc 的可执行文件应该链接到 /tools/lib64/ld-linux-x86-64.so.2 才对。那么,这就是问题所在

原因:在 5.9. Binutils-2.27 - Pass 2 和 5.10. GCC-6.2.0 - Pass 2 中,configure 前的 CC CXX AR RANLIB 等变量未设置,导致调用了默认编译器(即 host 上的gcc ),进而链接了 host 上的 glibc  , 而在 chroot 环境中因为 glibc 还没有安装,找不到库文件,于是得到一条迷惑性的提示 Command not found 。
分析:5.7. Glibc-2.24 和 5.10. GCC-6.2.0 - Pass 2 末尾的 dummy.c 测试都没问题,5.10 以后的(待续)

05 代码样本(可能已过时,github上有更新) 


#!/bin/bash
#
# handle all LFS jobs before chroot, the chrooted part named lfs2.bin.sh can be found in same directory.
############### Variables ###############
# disalbe hash
set +h
#umask 022
# set some env
PATH=/tools/bin:/bin:/usr/bin # must take first place after clean env, otherwise uname and set have no path
LFS=/mnt/lfs
LFS_TGT=$(uname -m)-lfs-linux-gnu
LC_ALL=POSIX
export PATH LFS LC_ALL LFS_TGT # must be done, subprocess such as ./configure need them
# temporary tools, *_p2 means pass 2, gcc_p3 is libstdc++
tmptools="binutils gcc linux glibc gcc_p3 binutils_p2 gcc_p2 tcl expect dejagnu check ncurses bash bzip2 coreutils diffutils file findutils gawk gettext grep gzip m4 make patch perl sed tar texinfo util-linux xz"
#tmptools="binutils gcc linux glibc gcc_p3 binutils_p2"
#tmptools="gcc_p2 tcl expect dejagnu check ncurses bash bzip2 coreutils diffutils file findutils gawk gettext grep gzip m4 make patch perl sed tar texinfo util-linux xz"
############### Functions ###############
check_host_env(){
[ -z "$LFS" ] && echo "\$LFS not set, check 2.6. Setting The \$LFS Variable to get detail" && exit
echo "############## Printing block list ##############"
echo
lsblk
echo
echo "############### RE-CHECK the list ###############"
echo
read -p "Are you sure ALL LFS partions were mounted correctly? [Y/n]" i
[ "$i" == "n" -o "$i" == "N" ] && exit
}
# 3.1. download packages
downloadpacks() {
check_host_env
wget http://www.linuxfromscratch.org/lfs/view/stable/wget-list --directory-prefix=$LFS/sources
wget http://www.linuxfromscratch.org/lfs/view/stable/md5sums --directory-prefix=$LFS/sources
wget --input-file=$LFS/sources/wget-list --continue --directory-prefix=$LFS/sources
# check packages md5sums
cd $LFS/sources
md5sum -c $LFS/sources/md5sums
}
# 4.2. Creating the $LFS/tools Directory
temptooldir(){
mkdir -pv $LFS/tools
# and link $LFS/tools to /tools
echo "link $LFS/tools to /tools, password needed"
sudo ln -sf $LFS/tools /
}
# called by extract_compile(), log configure, make and make install status code
report(){
if [ "$?" == "0" ]; then
echo `date +"%b %d %T"` $1 $2 complete! | tee -a $LFS/sources/log
else
echo `date +"%b %d %T"` $1 $2 have some problem!!! | tee -a $LFS/sources/log
fi
read -p "shall we continue? [Y/n]" i
[ "$i" == "n" -o "$i" == "N" ] && exit
}
# the main compile procedure
extract_compile(){
cd $LFS/sources
jj=$1 j=${jj%_p*}
configure="./configure" conf_argu= make="make" make_check= make_install="make install"
unset CC AR RANLIB CXX PKG_CONFIG EMACS #exported in binutils gcc pass 2 check gettext, unset before new loop
rm $j*/ -rf
tar xf $j*tar*
cd $j*/
case $jj in
gcc|gcc_p2)
tar xf ../mpfr*tar*
mv mpfr* mpfr
tar xf ../gmp*tar*
mv gmp* gmp
tar xf ../mpc*tar*
mv mpc* mpc
for file in $(find gcc/config -name linux64.h -o -name linux.h -o -name sysv4.h)
do
cp -uv $file{,.orig}
sed -e '[email protected]/lib\(64\)\?\(32\)\?/[email protected]/tools&@g' -e '[email protected]/[email protected]/[email protected]' $file.orig > $file
echo '
#undef STANDARD_STARTFILE_PREFIX_1
#undef STANDARD_STARTFILE_PREFIX_2
#define STANDARD_STARTFILE_PREFIX_1 "/tools/lib/"
#define STANDARD_STARTFILE_PREFIX_2 ""' >> $file
touch $file.orig
done
;;&
dejagnu|check|diffutils|file|findutils|gawk|grep|gzip|m4|make|patch|sed|tar|texinfo|xz)
# 5.1.3 dejagnu, "make check" after "make install", different?
make_check="make check"
;;&
binutils)
echo "compiling binutils, pass 1, 1 SBU, 3 min on i5-4200U"
conf_argu="--prefix=/tools --with-sysroot=$LFS --with-lib-path=/tools/lib --target=$LFS_TGT --disable-nls --disable-werror"
configure=../configure
;;
gcc)
echo "compiling gcc, pass 1, 8.3 SBU"
conf_argu="--target=$LFS_TGT --prefix=/tools --with-glibc-version=2.11 --with-sysroot=$LFS --with-newlib --without-headers --with-local-prefix=/tools --with-native-system-header-dir=/tools/include --disable-nls --disable-shared --disable-multilib --disable-decimal-float --disable-threads --disable-libatomic --disable-libgomp --disable-libmpx --disable-libquadmath --disable-libssp --disable-libvtv --disable-libstdcxx --enable-languages=c,c++"
configure=../configure
;;
linux)
echo "compiling headers, <0.1 SBU"
conf_argu=""
configure="echo I don't need configure..."
make="make mrproper"
make_install="make INSTALL_HDR_PATH=dest headers_install"
;;
glibc)
echo "compiling glibc, 4 SBU"
conf_argu="--prefix=/tools --host=$LFS_TGT --build=$(../scripts/config.guess) --enable-kernel=2.6.32 --with-headers=/tools/include libc_cv_forced_unwind=yes libc_cv_c_cleanup=yes"
configure=../configure
;;
gcc_p3)
echo "compiling libstdc++, 0.4 SBU"
conf_argu="--host=$LFS_TGT --prefix=/tools --disable-multilib --disable-nls --disable-libstdcxx-threads --disable-libstdcxx-pch --with-gxx-include-dir=/tools/$LFS_TGT/include/c++/6.2.0"
configure=../libstdc++-v3/configure
;;
binutils_p2)
echo "compiling binutils, pass 2, 1.1 SBU"
conf_argu="--prefix=/tools --disable-nls --disable-werror --with-lib-path=/tools/lib --with-sysroot"
configure=../configure
export CC=$LFS_TGT-gcc AR=$LFS_TGT-ar RANLIB=$LFS_TGT-ranlib
;;
gcc_p2)
echo "compiling gcc, pass 2, 11 SBU"
conf_argu="--prefix=/tools --with-local-prefix=/tools --with-native-system-header-dir=/tools/include --enable-languages=c,c++ --disable-libstdcxx-pch --disable-multilib --disable-bootstrap --disable-libgomp"
configure=../configure
cat gcc/limitx.h gcc/glimits.h gcc/limity.h > `dirname $($LFS_TGT-gcc -print-libgcc-file-name)`/include-fixed/limits.h
export CC=$LFS_TGT-gcc CXX=$LFS_TGT-g++ AR=$LFS_TGT-ar RANLIB=$LFS_TGT-ranlib
;;
tcl)
echo "compiling tcl, 0.4 SBU"
conf_argu="--prefix=/tools"
cd unix
;;
expect)
echo "compiling expect, 0.1 SBU"
conf_argu="--prefix=/tools --with-tcl=/tools/lib --with-tclinclude=/tools/include"
cp -v configure{,.orig}
sed 's:/usr/local/bin:/bin:' configure.orig > configure
make_install="make SCRIPTS="" install"
;;
dejagnu)
echo "compiling dejagnu, <0.1 SBU"
conf_argu="--prefix=/tools"
make="echo I don't need make..."
;;
check)
echo "compiling check, 0.1 SBU"
conf_argu="--prefix=/tools"
export PKG_CONFIG=
;;
ncurses)
echo "compiling ncurses, 0.5 SBU"
conf_argu="--prefix=/tools --with-shared --without-debug --without-ada --enable-widec --enable-overwrite"
sed -i s/mawk// configure
;;
bash)
echo "compiling bash, 0.4 SBU"
conf_argu="--prefix=/tools --without-bash-malloc"
;;
bzip2)
echo "compiling bzip2, <0.1 SBU"
conf_argu=""
configure="echo I don't need configure..."
make_install="make PREFIX=/tools install"
;;
coreutils)
echo "compiling coreutils, 0.6 SBU"
conf_argu="--prefix=/tools --enable-install-program=hostname"
make_check="make RUN_EXPENSIVE_TESTS=yes check"
;;
diffutils)
echo "compiling diffutils, 0.2 SBU"
conf_argu="--prefix=/tools"
;;
file)
echo "compiling file, 0.1 SBU"
conf_argu="--prefix=/tools"
;;
findutils)
echo "compiling findutils, 0.3 SBU"
conf_argu="--prefix=/tools"
;;
gawk)
echo "compiling gawk, 0.2 SBU"
conf_argu="--prefix=/tools"
;;
gettext)
echo "compiling gettext, 0.9 SBU"
conf_argu="--prefix=/tools --disable-shared"
cd gettext-tools
EMACS="no" $configure $conf_argu; report $jj configure
make -C gnulib-lib
make -C intl pluralx.c
make -C src msgfmt
make -C src msgmerge
make -C src xgettext
cp -v src/{msgfmt,msgmerge,xgettext} /tools/bin
return
;;
grep)
echo "compiling grep, 0.2 SBU"
conf_argu="--prefix=/tools"
;;
gzip)
echo "compiling gzip, 0.1 SBU"
conf_argu="--prefix=/tools"
;;
m4)
echo "compiling m4, 0.2 SBU"
conf_argu="--prefix=/tools"
;;
make)
echo "compiling make, 0.1 SBU"
conf_argu="--prefix=/tools --without-guile"
;;
patch)
echo "compiling patch, 0.2 SBU"
conf_argu="--prefix=/tools"
;;
perl)
echo "compiling perl, 1.3 SBU"
conf_argu=""
sh Configure -des -Dprefix=/tools -Dlibs=-lm
make; report $jj make
cp -v perl cpan/podlators/scripts/pod2man /tools/bin
mkdir -pv /tools/lib/perl5/5.24.0
cp -Rv lib/* /tools/lib/perl5/5.24.0
return
;;
sed)
echo "compiling sed, 0.1 SBU"
conf_argu="--prefix=/tools"
;;
tar)
echo "compiling tar, 0.3 SBU"
conf_argu="--prefix=/tools"
;;
texinfo)
echo "compiling texinfo, 0.2 SBU"
conf_argu="--prefix=/tools"
;;
util-linux)
echo "compiling util-linux, 0.8 SBU"
conf_argu="--prefix=/tools --without-python --disable-makeinstall-chown --without-systemdsystemunitdir PKG_CONFIG="""
;;
xz)
echo "compiling xz, 0.2 SBU"
conf_argu="--prefix=/tools"
;;
esac
case $j in
binutils|gcc|glibc)
# binutils binutils_p2 gcc gcc_p2 gcc_3 glibc, so use $j
rm build -rf; mkdir build; cd build
;;
esac
$configure $conf_argu; report $jj configure
$make; report $jj make
#$make_check; report $jj make-check
# something need to do after make but before make install
case $jj in
binutils)
[ `uname -m` == "x86_64" ] && mkdir -p /tools/lib && ln -sf lib /tools/lib64
;;
tcl)
#TZ=UTC make test
;;
expect)
make test
;;
bash)
make tests
;;
esac
$make_install; report $jj make-install
# something need to do after make install
case $jj in
linux)
cp -r dest/include/* /tools/include
;;
binutils_p2)
make -C ld clean
make -C ld LIB_PATH=/usr/lib:/lib
cp -v ld/ld-new /tools/bin
;;
gcc_p2)
ln -sf gcc /tools/bin/cc
;;
tcl)
chmod -v u+w /tools/lib/libtcl8.6.so
make install-private-headers
ln -sf tclsh8.6 /tools/bin/tclsh
;;
bash)
ln -sf bash /tools/bin/sh
;;
esac
tree -I sources $LFS > $LFS/sources/dir_tree_after_$jj
}
stripping(){
strip --strip-debug /tools/lib/*
/usr/bin/strip --strip-unneeded /tools/{,s}bin/*
rm -rf /tools/{,share}/{info,man,doc}
}
##### Chapter 6. Installing Basic System Software
# 6.2.Preparing Virtual Kernel File Systems
virtualfs(){
mkdir -pv $LFS/{dev,proc,sys,run}
# 6.2.1. Creating Initial Device Nodes
sudo mknod -m 600 $LFS/dev/console c 5 1
sudo mknod -m 666 $LFS/dev/null c 1 3
# 6.2.2. Mounting and Populating /dev
sudo mount -v --bind /dev $LFS/dev
# 6.2.3. Mounting Virtual Kernel File Systems
sudo mount -vt devpts devpts $LFS/dev/pts -o gid=5,mode=620
sudo mount -vt proc proc $LFS/proc
sudo mount -vt sysfs sysfs $LFS/sys
sudo mount -vt tmpfs tmpfs $LFS/run
if [ -h $LFS/dev/shm ]; then
mkdir -pv $LFS/$(readlink $LFS/dev/shm)
fi
}
# 6.3. Package Management
linkpackmanage(){
# 6.3.2. Package Management Techniques
./configure --prefix=/usr/pkg/libfoo/1.1
make
make install
./configure --prefix=/usr
make
make DESTDIR=/usr/pkg/libfoo/1.1 install
}
# 6.4. Entering the Chroot Environment
nowletschroot(){
echo "going to chroot, password needed"
sudo chroot "$LFS" /tools/bin/env -i HOME=/root TERM="$TERM" PS1='\u:\w\$ ' PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin /tools/bin/bash --login +h
}
############### Main Part ###############
case $1 in
-d)
downloadpacks
;;
-c)
check_host_env
;;
-s)
stripping
;;
-v)
virtualfs
;;
-r)
nowletschroot
;;
"")
check_host_env
temptooldir
for i in $tmptools
do
extract_compile $i
done
;;
*)
extract_compile $1
;;
esac

Tags

Add new comment

Plain text

  • No HTML tags allowed.
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.