# Error when run pylith on the cluster with PBS **URL:** \ **Category:** PyLith\ **Created:** [June 18, 2021, 1:22pm UTC](https://community.geodynamics.org/t/error-when-run-pylith-on-the-cluster-with-pbs/1884 "2021-06-18T13:22:38Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![PengZhai-07](https://yyz2.discourse-cdn.com/flex036/user_avatar/community.geodynamics.org/pengzhai-07/32/673_2.png) [@PengZhai-07](https://community.geodynamics.org/u/PengZhai-07)\ **Post date:** [June 18, 2021, 1:22pm UTC](https://community.geodynamics.org/t/error-when-run-pylith-on-the-cluster-with-pbs/1884/1 "2021-06-18T13:22:38Z")
Hello, I have complied the pylith on the cluster successfully. But when I try to run it using qsub, some errors happened. The following is the standard error information: Traceback (most recent call last): File “/home/node/pylith/bin/pylith”, line 25, in from pylith.apps.PyLithApp import PyLithApp ImportError: No module named pylith.apps.PyLithApp It seems that the program could’t find the location of PyLithApp. However, before submitting the task, I have excute the command “source ${HOME}/build/pylith/setup.sh” The following shows my qbs file: #!/bin/bash #PBS -N case01 #PBS -l nodes=cu01:ppn=12+cu02:ppn=12+cu03:ppn=12+cu04:ppn=12 #PBS -l walltime=480:00:00 #PBS -q batch1 #PBS -e my.err #PBS -o my.out EXEC=/home/node/pylith/bin/pylith INPUT=/home/node/pengzhai/script/step02\_7.cfg MONITOR=/home/node/pengzhai/script/step02\_7.log NP=48 nohup {EXEC} {INPUT} --nodes=${NP} So how should I configure my qbs file and figure out this problem? ---
**Author:** ![baagaard](https://yyz2.discourse-cdn.com/flex036/user_avatar/community.geodynamics.org/baagaard/32/615_2.png) [@baagaard](https://community.geodynamics.org/u/baagaard)\ **Post date:** [June 18, 2021, 6:03pm UTC](https://community.geodynamics.org/t/error-when-run-pylith-on-the-cluster-with-pbs/1884/2 "2021-06-18T18:03:58Z")
The problem is that the environment you set when you submit the job to the queue is not being used when the job actually starts.See the -V option of the qsub command: ```auto -V Declares that all environment variables in the qsub command's environment are to be exported to the batch job. ``` Note that PyLith can automatically submit a job to a PBS batch system. See the PyLith manual and `pyre/schedulers/SchedulerPBS.py` (part of the pyre package). ---
**Author:** ![PengZhai-07](https://yyz2.discourse-cdn.com/flex036/user_avatar/community.geodynamics.org/pengzhai-07/32/673_2.png) [@PengZhai-07](https://community.geodynamics.org/u/PengZhai-07)\ **Post date:** [June 19, 2021, 11:17am UTC](https://community.geodynamics.org/t/error-when-run-pylith-on-the-cluster-with-pbs/1884/3 "2021-06-19T11:17:44Z")
Thanks for your advice! I can excute “pylith step02\_7.cfg” in the command line successfully. But when I add the following four lines into the pylithapp.cfg file: [pylithapp] scheduler = pbs [pylithapp.launcher] command = mpirun -np ${nodes} -machinefile hosts and I run these in the command line: pylith step02\_7.cfg --job.queue=batch1 --job.name=step02\_7 --job.stdout=step02\_7.log --job.stderr=step02\_7.err --job.walltime=400\*hour --nodes=48 --scheduler.ppn=12 It deosn’t work and the step02\_7.err shows these information: Traceback (most recent call last): File “”, line 1, in File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/ **init**.py”, line 19, in from Shell import Shell File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/Shell.py”, line 25, in class Shell(Configurable): File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/Shell.py”, line 39, in Shell from Preprocessor import Preprocessor File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/Preprocessor.py”, line 17, in import os, getpass, socket File “/home/node/pylith/lib/python2.7/socket.py”, line 47, in import \_socket ImportError: /home/node/pylith/lib/python2.7/lib-dynload/\_socket.so: undefined symbol: PyUnicodeUCS2\_FromFormat So why this happened? 🤔 My hosts file includes: cu01 cu02 cu03 cu04 And using the qsub-options function, I can receieve the email: PBS Job Id: 1209.mu01 Job Name: step02\_7 Exec host: cu04/0-11+cu03/0-11+cu02/0-11+cu01/0-11 Execution terminated Exit\_status=1 resources\_used.cput=00:00:00 resources\_used.vmem=0kb resources\_used.walltime=00:00:01 resources\_used.mem=0kb resources\_used.energy\_used=0 Error\_Path: mu01:/home/node/pengzhai/script/step02\_7.err Output\_Path: mu01:/home/node/pengzhai/script/step02\_7.log ---
**Author:** ![baagaard](https://yyz2.discourse-cdn.com/flex036/user_avatar/community.geodynamics.org/baagaard/32/615_2.png) [@baagaard](https://community.geodynamics.org/u/baagaard)\ **Post date:** [June 20, 2021, 2:43am UTC](https://community.geodynamics.org/t/error-when-run-pylith-on-the-cluster-with-pbs/1884/4 "2021-06-20T02:43:20Z")
It still looks like your environment is not setup consistently from when you built PyLith and run it interactively versus run on a compute node. You can check this by doing `ldd /home/node/pylith/lib/python2.7/lib-dynload/_socket.so` when you are logged in and when you have a session on a compute node. You want to make sure all libraries are found and the paths are the same. ---
**Author:** ![PengZhai-07](https://yyz2.discourse-cdn.com/flex036/user_avatar/community.geodynamics.org/pengzhai-07/32/673_2.png) [@PengZhai-07](https://community.geodynamics.org/u/PengZhai-07)\ **Post date:** [June 21, 2021, 9:28am UTC](https://community.geodynamics.org/t/error-when-run-pylith-on-the-cluster-with-pbs/1884/5 "2021-06-21T09:28:22Z")
Hi Brad, ![pkgconfig](https://canada1.discourse-cdn.com/flex036/uploads/geodynamics/original/1X/61d35399a73f245fd7064527fa3f1a84a6bec698.jpeg) I compile Pylith today again and I meet a mistake when installing the h5py. It seems that there is something wrong with the pkgconfig.py. And I found that the pkgconfig-1.5.4 doesn’t support python2 anymore from [pkgconfig · PyPI](https://pypi.org/project/pkgconfig/) ! ![图片](https://canada1.discourse-cdn.com/flex036/uploads/geodynamics/original/1X/7c2180662550b55154d4e7e844a2aab48a1ce7e4.png) Then I had a try to excute the followings before continuing to install h5py > pip install cftime==1.1.1 > pip install Cython==0.29.16 > pip install pkgconfig==1.5.0 After that I continued to ‘make’, It looks like everything is ok! However, I don’t know if this way is right. 😅 New error does occur when I run Pylith on multiple nodes as before, and running Pylith on single node is ok. > Traceback (most recent call last): > File “”, line 1, in > File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/schedulers/ **init**.py”, line 53, in jobstart > kwds = kwds) > File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/ **init**.py”, line 41, in start > shell.run(\*\*kwds) > File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/Shell.py”, line 143, in run > method(\*args, \*\*kwds) > File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/SuperScript.py”, line 47, in execute > self.subscript = self.createSubscript(subscriptName) > File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/applications/AppRunner.py”, line 25, in createSubscript > cls = loadObject(name) > File “/home/node/pylith/lib/python2.7/site-packages/pythia-0.8.1.19-py2.7.egg/pyre/util/ **init**.py”, line 54, in loadObject > obj = **import** (module, globals(), globals(), [‘ **name** ’]) > File “/home/node/pylith/lib/python2.7/site-packages/pylith/apps/PyLithApp.py”, line 23, in > from PetscApplication import PetscApplication > File “/home/node/pylith/lib/python2.7/site-packages/pylith/apps/PetscApplication.py”, line 27, in > class PetscApplication(Application): > File “/home/node/pylith/lib/python2.7/site-packages/pylith/apps/PetscApplication.py”, line 41, in PetscApplication > from pylith.utils.PetscManager import PetscManager > File “/home/node/pylith/lib/python2.7/site-packages/pylith/utils/PetscManager.py”, line 29, in > import pylith.utils.petsc as petsc > File “/home/node/pylith/lib/python2.7/site-packages/pylith/utils/petsc.py”, line 28, in > \_petsc = swig\_import\_helper() > File “/home/node/pylith/lib/python2.7/site-packages/pylith/utils/petsc.py”, line 24, in swig\_import\_helper > \_mod = imp.load\_module(‘\_petsc’, fp, pathname, description) > ImportError: libsz.so.2: cannot open shared object file: No such file or directory Thanks, Zhai ---
**Author:** ![baagaard](https://yyz2.discourse-cdn.com/flex036/user_avatar/community.geodynamics.org/baagaard/32/615_2.png) [@baagaard](https://community.geodynamics.org/u/baagaard)\ **Post date:** [June 21, 2021, 2:31pm UTC](https://community.geodynamics.org/t/error-when-run-pylith-on-the-cluster-with-pbs/1884/6 "2021-06-21T14:31:33Z")
The PyLith installer v2.2.2-1 will install cftime==1.1.1 and Cython==0.29.16, so those versions should work. Once again, if you can build and run the tests but you get error messages about libraries not being found when you run on compute nodes, then your environment is not the same on the compute nodes as when you build. I recommend consulting with the system administrators for the cluster. They are in a much better position to help you than us.