Creating a single .exe file with pyinstaller by merging multiple python subprocess files

The script runs smoothly when executed as a .py file, but upon creating an executable file using pyinstaller, it enters an infinite loop. Interestingly, the program still operates normally despite the presence of the problematic code fragment in both the packaged and standard python file versions.

Question:

I have an inquiry that resembles this one: “Similar Question.” My GUI allows users to enter information, which other scripts utilize to function. For each button, I have four different scripts that I execute as sub-processes to prevent the main GUI from malfunctioning or displaying unresponsiveness. This is just a small snippet of the code since I used PAGE to create the GUI, and it’s lengthy.

###Main.py#####
import subprocess
def resource_path(relative_path):
    #I got this from another post to include images but I'm also using it to include the scripts"
    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = sys._MEIPASS
    except Exception:
        base_path = os.path.abspath(".")
    return os.path.join(base_path, relative_path)
Class aclass:
    def get_info(self):
        global ModelNumber, Serial,SpecFile,dateprint,Oper,outputfolder
        ModelNumber=self.Model.get()
        Serial=self.SerialNumber.get()
        outputfolder=self.TEntry2.get()
        SpecFile= self.Spec_File.get()
        return ModelNumber,Serial,SpecFile,outputfolder
    def First(self):
        aclass.get_info(self)                          #Where I use the resource path function
        First_proc = subprocess.Popen([sys.executable, resource_path('first.py'),str(ModelNumber),str(Serial),str(path),str(outputfolder)])
        First_proc.wait()
#####First.py#####
import numpy as np
import scipy 
from main import aclass
ModelNumber    = sys.argv[1]
Serial         = sys.argv[2]
path           = sys.argv[3]
path_save      = sys.argv[4]

This continues throughout my second, third, and fourth scripts.

In my spec file, I added:

a.datas +=[('first.py','C\path\to\script\first.py','DATA')]
a.datas +=[('main.py','C\path\to\script\main.py','DATA')]

The code is functional, but converting it to an .exe results in a crash as it fails to import first.py and its libraries (numpy, scipy, etc.). Despite trying to add it to a.datas and using runtime_hooks in the spec file, the issue persists. It is unclear whether the subprocess is the cause of the error. Are there any suggestions to resolve this issue?


Solution:

If it is not possible to reorganize your application in a way that eliminates the need for it, such as substituting

multiprocessing

for

subprocess

, then there are three alternatives available.

  • To guarantee that the .exe includes the scripts, either have them as an executable zipfile or utilize

    pkg_resources

    . Then, duplicate the script to a temporary directory to enable its execution.
  • Create a wrapper script with multiple entry points that can function as your primary program and execute as individual scripts. Even though running an unpacked script is not possible, importing a module from it is feasible within a packed exe.
  • Employ

    pkg_resources

    once more to create a covering function that executes the script by loading it as a string and utilizing

    exec

    in its execution.

The second option is the most neat, however, it requires some effort. Although some of the work can be done using

setuptools

entrypoints, it is more challenging to explain this method compared to doing it manually. Therefore, I will demonstrate the manual approach.


Suppose that the code you are referring to appears as follows:

# main.py
import subprocess
import sys
spam, eggs = sys.argv[1], sys.argv[2]
subprocess.run([sys.executable, 'vikings.py', spam])
subprocess.run([sys.executable, 'waitress.py', spam, eggs])
# vikings.py
import sys
print(' '.join(['spam'] * int(sys.argv[1])))
# waitress.py
import sys
import time
spam, eggs = int(sys.argv[1]), int(sys.argv[2]))
if eggs > spam:
    print("You can't have more eggs than spam!")
    sys.exit(2)
print("Frying...")
time.sleep(2)
raise Exception("This sketch is getting too silly!")

So, you run it like this:

$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!

To achieve the objective, we aim to rearrange the system by implementing a script that examines the command-line arguments to determine the imports required. The most minimal alteration to accomplish this is as follows:

# main.py
import subprocess
import sys
if sys.argv[1][:2] == '--':
    script = sys.argv[1][2:]
    if script == 'vikings':
        import vikings
        vikings.run(*sys.argv[2:])
    elif script == 'waitress':
        import waitress
        waitress.run(*sys.argv[2:])
    else:
        raise Exception(f'Unknown script {script}')
else:
    spam, eggs = sys.argv[1], sys.argv[2]
    subprocess.run([sys.executable, __file__, '--vikings', spam])
    subprocess.run([sys.executable, __file__, '--waitress', spam, eggs])
# vikings.py
def run(spam):
    print(' '.join(['spam'] * int(spam)))
# waitress.py
import sys
import time
def run(spam, eggs):
    spam, eggs = int(spam), int(eggs)
    if eggs > spam:
        print("You can't have more eggs than spam!")
        sys.exit(2)
    print("Frying...")
    time.sleep(2)
    raise Exception("This sketch is getting too silly!")

And now:

$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!

Here are some modifications that you may want to contemplate in your everyday routine.

  • To avoid redundancy, we can use a single line of code with appropriate error handling instead of copying and pasting the same three lines for each script and typing the script name thrice. For instance, we can use

    __import__(sys.argv[1][2:]).run(sys.argv[2:])

    as a replacement.
  • Utilize

    argparse

    in place of this makeshift condition for the initial argument. If you’re already transmitting non-
    trivial arguments
    to the scripts, you may already be employing

    argparse

    or a substitute.
  • To enable direct testing of scripts during development, simply include a block of

    if __name__ == '__main__':

    in each script that invokes

    run(sys.argv[1:])

    .

I refrained from doing any of these actions as they would hinder the idea in this insignificant illustration.


The documentation serves well as a review for those who have already completed the task, but falls short as a tutorial and explanation. Attempting to create a tutorial that surpasses the expertise of the PyPA experts may be too ambitious for a simple Stack Overflow answer.


Frequently Asked Questions