Nrthugu

I created a script (see below) to upgrade all my pip packages. I successfully executed my script via idle3, i.e. open the script using idle3 and pressing F5 to run the script as a module. However, I am not able to execute it in the terminal; got the below error. How do I overcome this error? Why does the import error happen in terminal but not in idle3?

$ python3 -m upgrade_pip_packages 

====================================================

UPGRADING ALL --USER PIP PACKAGES TO LATEST VERSION:

====================================================

Traceback (most recent call last):

  File "/usr/bin/pip", line 9, in <module>

    from pip import main

ImportError: cannot import name main

ERROR: Command 'pip list' returned non-zero exit status 1

My script : upgrade_pip_packages.py

#!/bin/python3

import subprocess

from pprint import pprint





def get_pkgs():

    try:

        cmd = 'pip list'

        completed = subprocess.run( cmd, shell=True, check=True,

                                    stdout=subprocess.PIPE )

    except subprocess.CalledProcessError as err:

        print( 'ERROR:', err )

    else:

        for line in completed.stdout.decode('utf-8').splitlines()[2:]:

            yield line





def update_pkgs(piplist):

    npackages = 0

    nupgrades = 0

    nerrors = 0

    upgradelist = 

    errorlist = 

    for i in piplist:

        npackages += 1

        pkgname, ver = i.split()

        print('n',pkgname)

        try:

            cmd = 'pip install --user {} --upgrade'.format(pkgname)

            completed = subprocess.run( cmd, shell=True, check=True,

                                        stdout=subprocess.PIPE )

        except subprocess.CalledProcessError as err:

            nerrors += 1

            errorlist.append(pkgname)

            print( 'ERROR: {}'.format(err) )

        else:

            for line in completed.stdout.decode('utf-8').splitlines():

                print(line)

                if 'Successfully installed' in line:

                    nupgrades +=1

                    upgradelist.append(pkgname)

    return npackages, nupgrades, nerrors, upgradelist, errorlist





def main():

    print('====================================================')

    print('UPGRADING ALL --USER PIP PACKAGES TO LATEST VERSION:')

    print('====================================================')

    pip_pkgs = get_pkgs() # created a generator

    npackages, nupgrades, nerrors, upgradelist, errorlist 

               = update_pkgs(pip_pkgs)

    print('nNo. of --user pip packages = {}'.format(npackages))

    print('No. of upgrades            = {}'.format(nupgrades))

    print('No. of upgrade errors      = {}'.format(nerrors))

    if upgradelist:

        print('Package(s) upgraded:')

        pprint(upgradelist)

    if errorlist:

        print('Package(s) with upgrade error:')

        pprint(errorlist)

    print()





if __name__ == '__main__':

    main()

Final upgrade_pip_packages.py.

I found the answer to my question. Essentially, my script had to:

import sys

and make the following amendments:

cmd = [sys.executable, '-m', 'pip', 'list'] #Change here

completed = subprocess.run( cmd,

                            #shell=True, #switch this off

                            check=True,

                            stdout=subprocess.PIPE )

and

cmd = [sys.executable, '-m', 'pip', 'install', '--user', pkgname, '--upgrade'] #Change here

completed = subprocess.run( cmd,

                            #shell=True, #switch this off

                            check=True,

                            stdout=subprocess.PIPE )

PyPA documentation explanation:

It’s recommended to write {sys.executable} rather than plain python in
order to ensure that commands are run in the Python installation
matching the currently running notebook (which may not be the same
Python installation that the python command refers to).

$ pip --version

pip 18.1 from ~/.local/lib/python3.5/site-packages/pip (python 3.5)

$ pip3 --version

pip 18.1 from ~/.local/lib/python3.5/site-packages/pip (python 3.5)