How To Run Bash Commands In Python

Below commands are useful for following version.

python --version

Python 3.7.4

There are different ways to run bash commands in Python. Lets start with os.system command.

How to use os.system to run Bash Command

import os

Once we have imported the os. We can use os.system and pass it bash command. Lets try ls -ld /home command

os.system("ls -ld /home")
0

The command is executed. We can't capture the output with os.system

How to use subprocess.check_output to run Bash Commands

To see the output of executed command. There is another way. We need to import Python package subprocess.

import subprocess
subprocess.check_output('ls -ld /home',shell=True, universal_newlines=True):
'drwxr-xr-x 14 root root 4096 Nov 28 16:12 /home\n'

How to use subprocess.run to run Bash Commands

To capture the output in a variable use the run method.

subprocess.run(['ls','-ld','/home'],check=True, stdout=subprocess.PIPE, universal_newlines=True):
CompletedProcess(args=['ls', '-ld', '/home'], returncode=0, stdout='drwxr-xr-x 14 root root 4096 Nov 28 16:12 /home\n')

We can also capture the output in a variable

output = subprocess.run(['ls','-ld','/home'],check=True, stdout=subprocess.PIPE, universal_newlines=True)

output can be printed using following command.

output.stdout
'drwxr-xr-x 14 root root 4096 Nov 28 16:12 /home\n'

We can also printout the errors using following command.

output.stderr

check=True options throw an error if underlying option throws an error. Lets try again.

subprocess.run(['ls','-ld','/home1'],check=True, stdout=subprocess.PIPE, universal_newlines=True)

CalledProcessError: Command '['ls', '-ld', '/home1']' returned non-zero exit status 2.

Lets try without check=True option

subprocess.run(['ls','-ld','/home1'], stdout=subprocess.PIPE, universal_newlines=True)
CompletedProcess(args=['ls', '-ld', '/home1'], returncode=2, stdout='')

Run Bash commands using Python Popen

Popen is used for complex commands, such as pipe commands in bash. Lets try a simple pipe command first.

p = subprocess.Popen(['ls','-ld','/home'],stderr=subprocess.PIPE, universal_newlines=True,stdout=subprocess.PIPE)
out,err = p.communicate()
print(out,err)
drwxr-xr-x 14 root root 4096 Nov 28 16:12 /home

p is a Python process. We can query it anytime. Also we can close it.

p.stdout.close()

Lets try accessing p again.

out,err = p.communicate()

We got the following error...

ValueError: Invalid file object: <_io.TextIOWrapper name=46 encoding='UTF-8'>

How to pipe commands together using Python Popen

Lets calculate number of files in the home directory.

p1 = subprocess.Popen(['ls','-lrt','/home'],stderr=subprocess.PIPE, universal_newlines=True,stdout=subprocess.PIPE)
p2 = subprocess.Popen(["wc", "-l"], stdin=p1.stdout, stdout=subprocess.PIPE)

Note, how we have to query p1.stdout to find the output.

Now lets look at the output.

output = p2.communicate()
print(output)
(b'23\n', None)

Wrap Up!

I have discussed above 5 ways to run Bash commands in Python. There is another utility called pysh which can be used to run Bash commands.

Table of Contents