Shell Tools and Scripting
Shell 工具和脚本
In this lecture, we will present some of the basics of using bash as a scripting language along with a number of shell tools that cover several of the most common tasks that you will be constantly performing in the command line.
在本讲座中,我们将介绍使用 bash 作为脚本语言的一些基础知识,以及一些 shell 工具,这些工具涵盖了你经常在命令行中执行的几项最常见任务。
Shell Scripting shell 脚本
So far we have seen how to execute commands in the shell and pipe them together.
However, in many scenarios you will want to perform a series of commands and make use of control flow expressions like conditionals or loops.
到目前为止,我们已经了解了如何在 shell 中执行命令并将它们连接在一起。不过,在许多情况下,你会希望执行一系列命令,并使用条件或循环等控制流表达式。
Shell scripts are the next step in complexity.
Most shells have their own scripting language with variables, control flow and its own syntax.
What makes shell scripting different from other scripting programming languages is that it is optimized for performing shell-related tasks.
Thus, creating command pipelines, saving results into files, and reading from standard input are primitives in shell scripting, which makes it easier to use than general purpose scripting languages.
For this section we will focus on bash scripting since it is the most common.
Shell 脚本是复杂性的下一步。大多数 shell 都有自己的脚本语言,包含变量、控制流和语法。shell 脚本与其他脚本编程语言的不同之处在于,它是为执行 shell 相关任务而优化的。因此,创建命令流水线、将结果保存到文件中以及从标准输入读取数据都是 shell 脚本的基本功能,这使得它比通用脚本语言更容易使用。在本节中,我们将重点讨论 bash 脚本,因为它是最常用的脚本语言。
To assign variables in bash, use the syntax foo=bar
and access the value of the variable with $foo
.
Note that foo = bar
will not work since it is interpreted as calling the foo
program with arguments =
and bar
.
In general, in shell scripts the space character will perform argument splitting. This behavior can be confusing to use at first, so always check for that.
要在 bash 中分配变量,请使用 foo=bar
语法,并用 $foo
访问变量值。请注意, foo = bar
不会起作用,因为它会被解释为调用带有参数 =
和 bar
的 foo
程序。 一般来说,在 shell 脚本中,空格字符会执行参数分割。刚开始使用时,这种行为可能会令人困惑,所以一定要检查一下。
Strings in bash can be defined with '
and "
delimiters, but they are not equivalent.
Strings delimited with '
are literal strings and will not substitute variable values whereas "
delimited strings will.
bash 中的字符串可以用 '
和 "
定界,但它们并不等同。用 '
分隔的字符串是字面字符串,不会替换变量值,而用 "
分隔的字符串则会。
foo=bar
echo "$foo"
# prints bar
echo '$foo'
# prints $foo
As with most programming languages, bash supports control flow techniques including if
, case
, while
and for
.
Similarly, bash
has functions that take arguments and can operate with them. Here is an example of a function that creates a directory and cd
s into it.
与大多数编程语言一样,bash 支持包括 if
、 case
、 while
和 for
在内的控制流技术。同样, bash
也有接收参数并能对参数进行操作的函数。下面是一个创建目录并将 cd
s 放入其中的函数示例。
mcd () {
mkdir -p "$1"
cd "$1"
}
Here $1
is the first argument to the script/function.
Unlike other scripting languages, bash uses a variety of special variables to refer to arguments, error codes, and other relevant variables. Below is a list of some of them. A more comprehensive list can be found here.
这里 $1
是脚本/函数的第一个参数。与其他脚本语言不同,bash 使用各种特殊变量来指代参数、错误代码和其他相关变量。下面是其中一些变量的列表。更全面的列表可以在这里找到。
$0
- Name of the script
$0
- 脚本名称$1
to$9
- Arguments to the script.$1
is the first argument and so on.
$1
至$9
- 脚本的参数。$1
是第一个参数,以此类推。$@
- All the arguments
$@
- 所有参数$#
- Number of arguments
$#
- 参数个数$?
- Return code of the previous command
$?
- 上一条命令的返回代码$$
- Process identification number (PID) for the current script
$$
- 当前脚本的进程标识号(PID)!!
- Entire last command, including arguments. A common pattern is to execute a command only for it to fail due to missing permissions; you can quickly re-execute the command with sudo by doingsudo !!
!!
- 最后一条命令的全部内容,包括参数。一种常见的模式是执行一条命令,但由于权限缺失而失败;你可以通过sudo !!
来快速使用 sudo 重新执行命令$_
- Last argument from the last command. If you are in an interactive shell, you can also quickly get this value by typingEsc
followed by.
orAlt+.
$_
- 最后一条命令的最后一个参数。如果在交互式 shell 中,也可以通过键入Esc
后跟.
或Alt+.
快速获得该值
Commands will often return output using STDOUT
, errors through STDERR
, and a Return Code to report errors in a more script-friendly manner.
The return code or exit status is the way scripts/commands have to communicate how execution went.
A value of 0 usually means everything went OK; anything different from 0 means an error occurred.
命令通常使用 STDOUT
返回输出,通过 STDERR
返回错误,并使用返回代码以更方便脚本的方式报告错误。返回代码或退出状态是脚本/命令交流执行情况的一种方式。返回值为 0 通常表示一切正常;与 0 不同的值则表示发生了错误。
Exit codes can be used to conditionally execute commands using &&
(and operator) and ||
(or operator), both of which are short-circuiting operators. Commands can also be separated within the same line using a semicolon ;
.
The true
program will always have a 0 return code and the false
command will always have a 1 return code.
Let’s see some examples
退出代码可以使用 &&
(和运算符)和 ||
(或运算符)有条件地执行命令,这两种运算符都是短路运算符。命令也可以使用分号 ;
在同一行内分隔。 true
程序的返回代码总是 0,而 false
命令的返回代码总是 1。我们来看几个例子
false || echo "Oops, fail"
# Oops, fail
true || echo "Will not be printed"
#
true && echo "Things went well"
# Things went well
false && echo "Will not be printed"
#
true ; echo "This will always run"
# This will always run
false ; echo "This will always run"
# This will always run
Another common pattern is wanting to get the output of a command as a variable. This can be done with command substitution.
Whenever you place $( CMD )
it will execute CMD
, get the output of the command and substitute it in place.
For example, if you do for file in $(ls)
, the shell will first call ls
and then iterate over those values.
A lesser known similar feature is process substitution, <( CMD )
will execute CMD
and place the output in a temporary file and substitute the <()
with that file’s name. This is useful when commands expect values to be passed by file instead of by STDIN. For example, diff <(ls foo) <(ls bar)
will show differences between files in dirs foo
and bar
.
另一种常见的模式是将命令的输出作为变量。这可以通过命令替换来实现。每当你执行 $( CMD )
命令时,它就会执行 CMD
命令,获取命令的输出并将其替换到位。例如,如果你执行 for file in $(ls)
,shell 会首先调用 ls
,然后遍历这些值。一个鲜为人知的类似功能是进程替换, <( CMD )
会执行 CMD
,并将输出放到一个临时文件中,然后用该文件名替换 <()
。当命令希望通过文件而不是 STDIN 传递值时,这个功能非常有用。例如, diff <(ls foo) <(ls bar)
会显示 foo
和 bar
目录下文件的差异。
Since that was a huge information dump, let’s see an example that showcases some of these features. It will iterate through the arguments we provide, grep
for the string foobar
, and append it to the file as a comment if it’s not found.
上面的信息量很大,下面我们来看一个例子来展示其中的一些功能。它将遍历我们提供的参数 grep
以查找字符串 foobar
,如果没有找到,就将其作为注释追加到文件中。
#!/bin/bash
echo "Starting program at $(date)" # Date will be substituted
echo "Running program $0 with $# arguments with pid $$"
for file in "$@"; do
grep foobar "$file" > /dev/null 2> /dev/null
# When pattern is not found, grep has exit status 1
# We redirect STDOUT and STDERR to a null register since we do not care about them
if [[ $? -ne 0 ]]; then
echo "File $file does not have any foobar, adding one"
echo "# foobar" >> "$file"
fi
done
In the comparison we tested whether $?
was not equal to 0.
Bash implements many comparisons of this sort - you can find a detailed list in the manpage for test
.
When performing comparisons in bash, try to use double brackets [[ ]]
in favor of simple brackets [ ]
. Chances of making mistakes are lower although it won’t be portable to sh
. A more detailed explanation can be found here.
在比较中,我们测试了 $?
是否不等于 0。Bash 实现了许多类似的比较,你可以在 test
的 manpage 中找到详细列表。在 bash 中进行比较时,尽量使用双括号 [[ ]]
而不是简单的括号 [ ]
。虽然不能移植到 sh
,但犯错的几率会降低。这里有更详细的解释。
When launching scripts, you will often want to provide arguments that are similar. Bash has ways of making this easier, expanding expressions by carrying out filename expansion. These techniques are often referred to as shell globbing.
在启动脚本时,你经常需要提供相似的参数。Bash 有办法让这一切变得更容易,通过执行文件名扩展来扩展表达式。这些技术通常被称为 shell globbing。
- Wildcards - Whenever you want to perform some sort of wildcard matching, you can use
?
and*
to match one or any amount of characters respectively. For instance, given filesfoo
,foo1
,foo2
,foo10
andbar
, the commandrm foo?
will deletefoo1
andfoo2
whereasrm foo*
will delete all butbar
.
通配符 - 当你想执行某种通配符匹配时,可以使用?
和*
分别匹配一个或任意数量的字符。例如,给定文件foo
,foo1
,foo2
,foo10
和bar
,命令rm foo?
将删除foo1
和foo2
,而rm foo*
将删除除bar
以外的所有文件。 - Curly braces
{}
- Whenever you have a common substring in a series of commands, you can use curly braces for bash to expand this automatically. This comes in very handy when moving or converting files.
大括号{}
- 在一系列命令中,只要有一个共同的子串,就可以使用大括号让 bash 自动展开。这在移动或转换文件时非常方便。
convert image.{png,jpg}
# Will expand to
convert image.png image.jpg
cp /path/to/project/{foo,bar,baz}.sh /newpath
# Will expand to
cp /path/to/project/foo.sh /path/to/project/bar.sh /path/to/project/baz.sh /newpath
# Globbing techniques can also be combined
mv *{.py,.sh} folder
# Will move all *.py and *.sh files
mkdir foo bar
# This creates files foo/a, foo/b, ... foo/h, bar/a, bar/b, ... bar/h
touch {foo,bar}/{a..h}
touch foo/x bar/y
# Show differences between files in foo and bar
diff <(ls foo) <(ls bar)
# Outputs
# < x
# ---
# > y
Writing bash
scripts can be tricky and unintuitive. There are tools like shellcheck that will help you find errors in your sh/bash scripts.
编写 bash
脚本可能既棘手又不直观。shellcheck 等工具可以帮助你发现 sh/bash 脚本中的错误。
Note that scripts need not necessarily be written in bash to be called from the terminal. For instance, here’s a simple Python script that outputs its arguments in reversed order:
请注意,从终端调用脚本并不一定要用 bash 编写。例如,下面是一个简单的 Python 脚本,它以相反的顺序输出参数:
#!/usr/local/bin/python
import sys
for arg in reversed(sys.argv[1:]):
print(arg)
The kernel knows to execute this script with a python interpreter instead of a shell command because we included a shebang line at the top of the script.
It is good practice to write shebang lines using the env
command that will resolve to wherever the command lives in the system, increasing the portability of your scripts. To resolve the location, env
will make use of the PATH
environment variable we introduced in the first lecture.
For this example the shebang line would look like #!/usr/bin/env python
.
内核知道该脚本是用 python 解释器执行的,而不是用 shell 命令,因为我们在脚本顶端加入了 Shebang 行。使用 env
命令编写 Shebang 行是一种好的做法,它能将命令解析到系统中的任何位置,从而提高脚本的可移植性。为了确定位置, env
将使用我们在第一讲中介绍的 PATH
环境变量。在本例中,shebang 行看起来像 #!/usr/bin/env python
.
Some differences between shell functions and scripts that you should keep in mind are:
shell 函数和脚本之间的一些区别值得注意:
- Functions have to be in the same language as the shell, while scripts can be written in any language. This is why including a shebang for scripts is important.
函数必须使用与 shell 相同的语言,而脚本可以使用任何语言编写。因此,为脚本添加 Shebang 非常重要。 - Functions are loaded once when their definition is read. Scripts are loaded every time they are executed. This makes functions slightly faster to load, but whenever you change them you will have to reload their definition.
函数在读取其定义时加载一次。而脚本在每次执行时都会被加载。这使得函数的加载速度稍快,但无论何时更改,都必须重新加载其定义。 - Functions are executed in the current shell environment whereas scripts execute in their own process. Thus, functions can modify environment variables, e.g. change your current directory, whereas scripts can’t. Scripts will be passed by value environment variables that have been exported using
export
函数在当前 shell 环境中执行,而脚本则在自己的进程中执行。因此,函数可以修改环境变量,例如更改当前目录,而脚本则不能。脚本将通过使用export
导出的环境变量值传递。 - As with any programming language, functions are a powerful construct to achieve modularity, code reuse, and clarity of shell code. Often shell scripts will include their own function definitions.
与其他编程语言一样,函数是实现模块化、代码重用和 shell 代码清晰度的强大结构。shell 脚本通常会包含自己的函数定义。
Shell Tools shell 工具
Finding how to use commands
查找如何使用命令
At this point, you might be wondering how to find the flags for the commands in the aliasing section such as ls -l
, mv -i
and mkdir -p
.
More generally, given a command, how do you go about finding out what it does and its different options?
You could always start googling, but since UNIX predates StackOverflow, there are built-in ways of getting this information.
说到这里,你可能想知道如何找到别名部分的命令标志,如 ls -l
、 mv -i
和 mkdir -p
。更笼统地说,给定一个命令,你该如何找出它的作用和不同的选项?你可以上网搜索,但由于 UNIX 早于 StackOverflow 的出现,因此有一些内置的方法可以获取这些信息。
As we saw in the shell lecture, the first-order approach is to call said command with the -h
or --help
flags. A more detailed approach is to use the man
command.
Short for manual, man
provides a manual page (called manpage) for a command you specify.
For example, man rm
will output the behavior of the rm
command along with the flags that it takes, including the -i
flag we showed earlier.
In fact, what I have been linking so far for every command is the online version of the Linux manpages for the commands.
Even non-native commands that you install will have manpage entries if the developer wrote them and included them as part of the installation process.
For interactive tools such as the ones based on ncurses, help for the commands can often be accessed within the program using the :help
command or typing ?
.
正如我们在 shell 讲座中所看到的,第一种方法是使用 -h
或 --help
标志调用上述命令。更详细的方法是使用 man
命令。作为 manual 的缩写, man
为你指定的命令提供了一个手册页面(称为 manpage)。例如, man rm
将输出 rm
命令的行为及其使用的标志,包括我们前面展示的 -i
标志。事实上,到目前为止,我为每条命令链接的都是该命令的 Linux 手册在线版本。即使是你安装的非本地命令,如果开发者编写了它们并将其作为安装过程的一部分,那么它们也会有 manpage 条目。对于交互式工具(如基于 ncurses 的工具),通常可以在程序中使用 :help
命令或键入 ?
访问命令帮助。
Sometimes manpages can provide overly detailed descriptions of the commands, making it hard to decipher what flags/syntax to use for common use cases.
TLDR pages are a nifty complementary solution that focuses on giving example use cases of a command so you can quickly figure out which options to use.
For instance, I find myself referring back to the tldr pages for tar
and ffmpeg
way more often than the manpages.
有时,手册会对命令进行过于详细的描述,这就很难解释在常用情况下应使用哪些标志/语法。TLDR 页面是一个很好的补充解决方案,它侧重于提供命令的示例用例,这样你就能快速找出应该使用哪些选项。例如,我发现自己查阅 tar
和 ffmpeg
的 TLDR 页面比查阅 manpages 更频繁。
Finding files 查找文件
One of the most common repetitive tasks that every programmer faces is finding files or directories.
All UNIX-like systems come packaged with find
, a great shell tool to find files. find
will recursively search for files matching some criteria. Some examples:
查找文件或目录是每个程序员最常面对的重复性工作之一。所有类 UNIX 系统都附带有 find
这个查找文件的优秀 shell 工具。 find
会递归搜索符合某些条件的文件。举几个例子
# Find all directories named src
find . -name src -type d
# Find all python files that have a folder named test in their path
find . -path '*/test/*.py' -type f
# Find all files modified in the last day
find . -mtime -1
# Find all zip files with size in range 500k to 10M
find . -size +500k -size -10M -name '*.tar.gz'
Beyond listing files, find can also perform actions over files that match your query.
This property can be incredibly helpful to simplify what could be fairly monotonous tasks.
除了列出文件外,find 还可以对符合查询条件的文件执行操作。这一特性对于简化可能相当单调的工作非常有帮助。
# Delete all files with .tmp extension
find . -name '*.tmp' -exec rm {} \;
# Find all PNG files and convert them to JPG
find . -name '*.png' -exec convert {} {}.jpg \;
Despite find
’s ubiquitousness, its syntax can sometimes be tricky to remember.
For instance, to simply find files that match some pattern PATTERN
you have to execute find -name '*PATTERN*'
(or -iname
if you want the pattern matching to be case insensitive).
You could start building aliases for those scenarios, but part of the shell philosophy is that it is good to explore alternatives.
Remember, one of the best properties of the shell is that you are just calling programs, so you can find (or even write yourself) replacements for some.
For instance, fd
is a simple, fast, and user-friendly alternative to find
.
It offers some nice defaults like colorized output, default regex matching, and Unicode support. It also has, in my opinion, a more intuitive syntax.
For example, the syntax to find a pattern PATTERN
is fd PATTERN
.
尽管 find
无处不在,但其语法有时却很难记住。例如,要简单地查找与 PATTERN
模式匹配的文件,你必须执行 find -name '*PATTERN*'
(或 -iname
,如果你希望模式匹配不区分大小写)。你可以开始为这些情况建立别名,但 shell 哲学的一部分就是探索替代方案。请记住,shell 最好的特性之一就是你只是在调用程序,所以你可以找到(甚至自己编写)一些程序的替代品。例如, fd
是 find
的一个简单、快速、用户友好的替代程序,它提供了一些不错的默认设置,如彩色输出、默认 regex 匹配和 Unicode 支持。在我看来,它的语法也更直观。例如,查找 PATTERN
模式的语法是 fd PATTERN
。
Most would agree that find
and fd
are good, but some of you might be wondering about the efficiency of looking for files every time versus compiling some sort of index or database for quickly searching.
That is what locate
is for.
locate
uses a database that is updated using updatedb
.
In most systems, updatedb
is updated daily via cron
.
Therefore one trade-off between the two is speed vs freshness.
Moreover find
and similar tools can also find files using attributes such as file size, modification time, or file permissions, while locate
just uses the file name.
A more in-depth comparison can be found here.
大多数人都认为 find
和 fd
很好,但有些人可能会问,每次查找文件的效率要比编制某种索引或数据库以便快速搜索的效率高。这就是 locate
的作用。在大多数系统中, updatedb
是通过 cron
每天更新的。因此,两者之间的一个权衡是速度与新鲜度。此外, find
和类似工具还可以使用文件大小、修改时间或文件权限等属性查找文件,而 locate
只使用文件名。更深入的比较请点击此处。
Finding code 查找代码
Finding files by name is useful, but quite often you want to search based on file content.
A common scenario is wanting to search for all files that contain some pattern, along with where in those files said pattern occurs.
To achieve this, most UNIX-like systems provide grep
, a generic tool for matching patterns from the input text.
grep
is an incredibly valuable shell tool that we will cover in greater detail during the data wrangling lecture.
通过文件名查找文件很有用,但很多时候你需要根据文件内容进行搜索。一种常见的情况是,想搜索包含某种模式的所有文件,以及该模式在这些文件中出现的位置。为了实现这一目的,大多数类 UNIX 系统都提供了 grep
,这是一个从输入文本中匹配模式的通用工具。 grep
是一个非常有价值的 shell 工具,我们将在数据处理讲座中详细介绍。
For now, know that grep
has many flags that make it a very versatile tool.
Some I frequently use are -C
for getting Context around the matching line and -v
for inverting the match, i.e. print all lines that do not match the pattern. For example, grep -C 5
will print 5 lines before and after the match.
When it comes to quickly searching through many files, you want to use -R
since it will Recursively go into directories and look for files for the matching string.
现在,我们要知道 grep
有许多标志,使其成为一个非常通用的工具。我经常使用的一些标志有: -C
,用于获取匹配行周围的上下文; -v
,用于反转匹配,即打印所有与模式不匹配的行。例如, grep -C 5
将打印匹配前后的 5 行。当需要快速搜索多个文件时,可以使用 -R
,因为它会递归进入目录,查找匹配字符串的文件。
But grep -R
can be improved in many ways, such as ignoring .git
folders, using multi CPU support, &c.
Many grep
alternatives have been developed, including ack, ag and rg.
All of them are fantastic and pretty much provide the same functionality.
For now I am sticking with ripgrep (rg
), given how fast and intuitive it is. Some examples:
但 grep -R
可以在很多方面进行改进,如忽略 .git
文件夹、使用多 CPU 支持等。目前已开发出许多 grep
替代方案,包括 ack、ag 和 rg。它们都非常出色,而且提供的功能基本相同。鉴于 ripgrep ( rg
) 的速度和直观性,我现在还是坚持使用它。举几个例子:
# Find all python files where I used the requests library
rg -t py 'import requests'
# Find all files (including hidden files) without a shebang line
rg -u --files-without-match "^#\!"
# Find all matches of foo and print the following 5 lines
rg foo -A 5
# Print statistics of matches (# of matched lines and files )
rg --stats PATTERN
Note that as with find
/fd
, it is important that you know that these problems can be quickly solved using one of these tools, while the specific tools you use are not as important.
请注意,与 find
/ fd
一样,重要的是你要知道这些问题可以用这些工具之一快速解决,而你使用的具体工具并不那么重要。
Finding shell commands 查找 shell 命令
So far we have seen how to find files and code, but as you start spending more time in the shell, you may want to find specific commands you typed at some point.
The first thing to know is that typing the up arrow will give you back your last command, and if you keep pressing it you will slowly go through your shell history.
到目前为止,我们已经了解了如何查找文件和代码,但随着你在 shell 中使用的时间越来越长,你可能会想要查找你曾输入过的特定命令。首先要知道的是,键入向上箭头会返回上一个命令,如果一直按下去,就会慢慢浏览 shell 历史记录。
The history
command will let you access your shell history programmatically.
It will print your shell history to the standard output.
If we want to search there we can pipe that output to grep
and search for patterns.
history | grep find
will print commands that contain the substring “find”.
history
命令可以让你以编程方式访问 shell 历史记录。它会将 shell 历史记录打印到标准输出。如果我们想在这里搜索,可以将输出导入 grep
并搜索模式。 history | grep find
将打印包含子串 "find "的命令。
In most shells, you can make use of Ctrl+R
to perform backwards search through your history.
After pressing Ctrl+R
, you can type a substring you want to match for commands in your history.
As you keep pressing it, you will cycle through the matches in your history.
This can also be enabled with the UP/DOWN arrows in zsh.
A nice addition on top of Ctrl+R
comes with using fzf bindings.
fzf
is a general-purpose fuzzy finder that can be used with many commands.
Here it is used to fuzzily match through your history and present results in a convenient and visually pleasing manner.
在大多数 shell 中,你可以使用 Ctrl+R
在历史记录中进行反向搜索。按下 Ctrl+R
后,您可以键入一个子字符串来匹配历史记录中的命令。当你持续按下它时,你将循环浏览历史记录中的匹配命令。这也可以通过 zsh 中的上/下箭头启用。在 Ctrl+R
的基础上,还可以使用 fzf 绑定。 fzf
是一个通用的模糊查找器,可与许多命令一起使用。在这里,它用于对历史记录进行模糊匹配,并以一种方便、视觉愉悦的方式呈现结果。
Another cool history-related trick I really enjoy is history-based autosuggestions.
First introduced by the fish shell, this feature dynamically autocompletes your current shell command with the most recent command that you typed that shares a common prefix with it.
It can be enabled in zsh and it is a great quality of life trick for your shell.
我非常喜欢的另一个与历史相关的酷技巧是基于历史的自动建议。这项功能最早由 fish shell 引入,它能动态地将当前 shell 命令与你最近输入的、前缀相同的命令进行自动补全。它可以在 zsh 中启用,是提高 shell 使用质量的绝佳技巧。
You can modify your shell’s history behavior, like preventing commands with a leading space from being included. This comes in handy when you are typing commands with passwords or other bits of sensitive information.
To do this, add HISTCONTROL=ignorespace
to your .bashrc
or setopt HIST_IGNORE_SPACE
to your .zshrc
.
If you make the mistake of not adding the leading space, you can always manually remove the entry by editing your .bash_history
or .zsh_history
.
你可以修改 shell 的历史记录行为,比如防止包含前导空格的命令。当你输入带有密码或其他敏感信息的命令时,这就派上用场了。要做到这一点,可以在 .bashrc
中添加 HISTCONTROL=ignorespace
或在 .zshrc
中添加 setopt HIST_IGNORE_SPACE
。 如果你犯了没有添加前导空格的错误,可以通过编辑 .bash_history
或 .zsh_history
手动删除条目。
Directory Navigation 目录导航
So far, we have assumed that you are already where you need to be to perform these actions. But how do you go about quickly navigating directories?
There are many simple ways that you could do this, such as writing shell aliases or creating symlinks with ln -s, but the truth is that developers have figured out quite clever and sophisticated solutions by now.
到目前为止,我们假设您已经在需要执行这些操作的地方了。但如何快速浏览目录呢?有许多简单的方法可以做到这一点,例如编写 shell 别名或使用 ln -s 创建符号链接,但事实上,开发人员现在已经找到了相当聪明和复杂的解决方案。
As with the theme of this course, you often want to optimize for the common case.
Finding frequent and/or recent files and directories can be done through tools like fasd
and autojump
.
Fasd ranks files and directories by frecency, that is, by both frequency and recency.
By default, fasd
adds a z
command that you can use to quickly cd
using a substring of a frecent directory. For example, if you often go to /home/user/files/cool_project
you can simply use z cool
to jump there. Using autojump, this same change of directory could be accomplished using j cool
.
正如本课程的主题一样,你通常希望针对常见情况进行优化。可以通过 fasd
和 autojump
等工具来查找经常访问和/或最近访问的文件和目录。Fasd 按频率对文件和目录进行排序,即同时按频率和最近性排序。默认情况下, fasd
会添加一个 z
命令,你可以用它来使用频率目录的子串快速 cd
。例如,如果你经常访问 /home/user/files/cool_project
,你可以直接使用 z cool
跳转到那里。使用自动跳转功能,同样的目录变更也可以用 j cool
来完成。
More complex tools exist to quickly get an overview of a directory structure: tree
, broot
or even full fledged file managers like nnn
or ranger
.
还有一些更复杂的工具可以快速浏览目录结构: tree
、 broot
甚至是完整的文件管理器,如 nnn
或 ranger
。
Exercises 练习
-
Read
man ls
and write anls
command that lists files in the following manner
读取man ls
并编写ls
命令,以下列方式列出文件- Includes all files, including hidden files
包含所有文件,包括隐藏文件 - Sizes are listed in human readable format (e.g. 454M instead of 454279954)
大小以人类可读格式列出(如 454M 而不是 454279954) - Files are ordered by recency
文件按新旧程度排序 - Output is colorized 输出为彩色
A sample output would look like this
输出示例如下-rw-r--r-- 1 user group 1.1M Jan 14 09:53 baz drwxr-xr-x 5 user group 160 Jan 14 09:53 . -rw-r--r-- 1 user group 514 Jan 14 06:42 bar -rw-r--r-- 1 user group 106M Jan 13 12:12 foo drwx------+ 47 user group 1.5K Jan 12 18:08 ..
- Includes all files, including hidden files
-
Write bash functions
marco
andpolo
that do the following. Whenever you executemarco
the current working directory should be saved in some manner, then when you executepolo
, no matter what directory you are in,polo
shouldcd
you back to the directory where you executedmarco
. For ease of debugging you can write the code in a filemarco.sh
and (re)load the definitions to your shell by executingsource marco.sh
.
编写 bash 函数marco
和polo
,实现以下功能。每当你执行marco
时,当前的工作目录应该以某种方式保存下来,然后当你执行polo
时,无论你在哪个目录,polo
都会cd
你回到执行marco
的目录。为了便于调试,你可以将代码写入marco.sh
文件,然后通过执行source marco.sh
将定义(重新)加载到 shell 中。 -
Say you have a command that fails rarely. In order to debug it you need to capture its output but it can be time consuming to get a failure run. Write a bash script that runs the following script until it fails and captures its standard output and error streams to files and prints everything at the end. Bonus points if you can also report how many runs it took for the script to fail.
假设您有一条很少失败的命令。为了调试它,你需要捕获它的输出,但获取失败运行结果可能很费时间。编写一个 bash 脚本,运行下面的脚本直到失败,并将其标准输出和错误流捕获到文件中,最后打印所有内容。如果还能报告脚本运行失败的次数,则可获得加分。#!/usr/bin/env bash n=$(( RANDOM % 100 )) if [[ n -eq 42 ]]; then echo "Something went wrong" >&2 echo "The error was using magic numbers" exit 1 fi echo "Everything went according to plan"
-
As we covered in the lecture
find
’s-exec
can be very powerful for performing operations over the files we are searching for. However, what if we want to do something with all the files, like creating a zip file? As you have seen so far commands will take input from both arguments and STDIN. When piping commands, we are connecting STDOUT to STDIN, but some commands liketar
take inputs from arguments. To bridge this disconnect there’s thexargs
command which will execute a command using STDIN as arguments. For examplels | xargs rm
will delete the files in the current directory.
正如我们在find
'-exec
的讲座中所介绍的那样,它在对我们要搜索的文件执行操作时非常强大。但是,如果我们想对所有文件做一些操作,比如创建一个压缩文件呢?正如你迄今为止所看到的,命令将从参数和 STDIN 两处接收输入。在使用管道连接命令时,我们会将 STDOUT 连接到 STDIN,但有些命令(如tar
)会从参数中获取输入。为了弥补这种脱节,我们可以使用xargs
命令,它将使用 STDIN 作为参数来执行命令。例如,ls | xargs rm
将删除当前目录下的文件。Your task is to write a command that recursively finds all HTML files in the folder and makes a zip with them. Note that your command should work even if the files have spaces (hint: check
-d
flag forxargs
).
你的任务是编写一条命令,递归查找文件夹中的所有 HTML 文件,并将它们压缩成一个 zip 文件。请注意,即使文件中有空格,您的命令也应能正常运行(提示:检查-d
标志是否为xargs
)。If you’re on macOS, note that the default BSD
find
is different from the one included in GNU coreutils. You can use-print0
onfind
and the-0
flag onxargs
. As a macOS user, you should be aware that command-line utilities shipped with macOS may differ from the GNU counterparts; you can install the GNU versions if you like by using brew.
如果你使用的是 macOS,请注意 BSD 默认的find
与 GNU coreutils 中的find
不同。你可以在find
上使用-print0
标志,在xargs
上使用-0
标志。作为 macOS 用户,你应该注意 macOS 附带的命令行实用程序可能与 GNU 对应版本不同;如果你喜欢,可以使用 brew 安装 GNU 版本。 -
(Advanced) Write a command or script to recursively find the most recently modified file in a directory. More generally, can you list all files by recency?
(高级)编写一条命令或脚本,递归查找目录中最近修改的文件。更一般地说,能否按最近修改次数列出所有文件?
Edit this page. 编辑本页。
Licensed under CC BY-NC-SA.
采用 CC BY-NC-SA 许可。