I have addressed problems with building the h5py and netCDF4 Python packages. I patched the installer to install Cython version 0.29.16 and cftime version 1.1.1. You can download the installer pylith-installer-2.2.2-1.tgz from the PyLith Installer releases page.
Hi Brad,
I am trying to compile pylith on macOS 10.13.6 with source version 2.2.2-1 which resolves the openssl bug shown in the discussion above.
Now I get the following error ...
creating build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying setuptools.egg-info/PKG-INFO → build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying setuptools.egg-info/SOURCES.txt → build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying setuptools.egg-info/dependency_links.txt → build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying setuptools.egg-info/entry_points.txt → build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying setuptools.egg-info/requires.txt → build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying setuptools.egg-info/top_level.txt → build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying setuptools.egg-info/zip-safe → build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
creating dist
creating ‘dist/setuptools-39.1.0-py2.7.egg’ and adding ‘build/bdist.macosx-10.13-x86_64/egg’ to it
removing ‘build/bdist.macosx-10.13-x86_64/egg’ (and everything under it)
Creating /Users/cyclo/pylith/lib/python2.7/site-packages/site.py
Processing setuptools-39.1.0-py2.7.egg
Copying setuptools-39.1.0-py2.7.egg to /Users/cyclo/pylith/lib/python2.7/site-packages
Adding setuptools 39.1.0 to easy-install.pth file
Installing easy_install script to /Users/cyclo/pylith/bin
Installing easy_install-2.7 script to /Users/cyclo/pylith/bin
Installed /Users/cyclo/pylith/lib/python2.7/site-packages/setuptools-39.1.0-py2.7.egg
Processing dependencies for setuptools==39.1.0
Finished processing dependencies for setuptools==39.1.0
/Users/cyclo/Pylith222_1/src/pylith/pylith-installer-2.2.2-1/bin/fetch.sh “/opt/local/bin/curl -L -O -o” numpy-1.14.3.tar.gz Index of /~buildbot/deps
Downloading numpy-1.14.3.tar.gz from Index of /~buildbot/deps.
Total Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4267k 100 4267k 0 0 1585k 0 0:00:02 0:00:02 --:–:-- 1589k
/usr/bin/tar -zxf numpy-1.14.3.tar.gz
cd numpy-1.14.3 &&
/Users/cyclo/pylith/bin/python setup.py install --prefix=/Users/cyclo/pylith &&
touch /Users/cyclo/Pylith222_1/installed_numpy
Running from numpy source directory.
Note: if you need reliable uninstall behavior, then install
with pip instead of using setup.py install:
-
pip install .(from a git repo or downloaded source
release) -
pip install numpy(last NumPy release on PyPi)
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 147, in
globals()[__func_name] = __get_hash(__func_name)
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 97, in __get_builtin_constructor
raise ValueError('unsupported hash type ’ + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 147, in
globals()[__func_name] = __get_hash(__func_name)
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 97, in __get_builtin_constructor
raise ValueError('unsupported hash type ’ + name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 147, in
globals()[__func_name] = __get_hash(__func_name)
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 97, in __get_builtin_constructor
raise ValueError('unsupported hash type ’ + name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 147, in
globals()[__func_name] = __get_hash(__func_name)
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 97, in __get_builtin_constructor
raise ValueError('unsupported hash type ’ + name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 147, in
globals()[__func_name] = __get_hash(__func_name)
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 97, in __get_builtin_constructor
raise ValueError(‘unsupported hash type ’ + name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 147, in
globals()[__func_name] = __get_hash(__func_name)
File “/Users/cyclo/pylith/lib/python2.7/hashlib.py”, line 97, in __get_builtin_constructor
raise ValueError(‘unsupported hash type ’ + name)
ValueError: unsupported hash type sha512
Traceback (most recent call last):
File “setup.py”, line 394, in
setup_package()
File “setup.py”, line 386, in setup_package
setup(**metadata)
File “/Users/cyclo/Pylith222_1/numpy-1.14.3/numpy/distutils/core.py”, line 135, in setup
config = configuration()
File “setup.py”, line 166, in configuration
config.add_subpackage(‘numpy’)
File “/Users/cyclo/Pylith222_1/numpy-1.14.3/numpy/distutils/misc_util.py”, line 1024, in add_subpackage
caller_level = 2)
File “/Users/cyclo/Pylith222_1/numpy-1.14.3/numpy/distutils/misc_util.py”, line 993, in get_subpackage
caller_level = caller_level + 1)
File “/Users/cyclo/Pylith222_1/numpy-1.14.3/numpy/distutils/misc_util.py”, line 930, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File “numpy/setup.py”, line 10, in configuration
config.add_subpackage(‘core’)
File “/Users/cyclo/Pylith222_1/numpy-1.14.3/numpy/distutils/misc_util.py”, line 1024, in add_subpackage
caller_level = 2)
File “/Users/cyclo/Pylith222_1/numpy-1.14.3/numpy/distutils/misc_util.py”, line 993, in get_subpackage
caller_level = caller_level + 1)
File “/Users/cyclo/Pylith222_1/numpy-1.14.3/numpy/distutils/misc_util.py”, line 930, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File “numpy/core/setup.py”, line 400, in configuration
check_api_version(C_API_VERSION, codegen_dir)
File “numpy/core/setup_common.py”, line 83, in check_api_version
curapi_hash, api_hash = get_api_versions(apiversion, codegen_dir)
File “numpy/core/setup_common.py”, line 74, in get_api_versions
curapi_hash = m.fullapi_hash(numpy_api.full_api)
File “numpy/core/code_generators/genapi.py”, line 482, in fullapi_hash
return hashlib.md5(’’.join(a).encode(‘ascii’)).hexdigest()
AttributeError: ‘module’ object has no attribute ‘md5’
make[3]: *** [numpy] Error 1
make[2]: *** [installed_numpy] Error 2
make[1]: *** [installed_h5py] Error 2
make: *** [installed_pylith] Error 2
Can you help me with that?
My guess is that you are trying to use the system OpenSSL library, which is incompatible or not found. You can use --enable-openssl when configuring the installer to have it build the OpenSSL library.
Note: Is there a reason you are not using the maxOS binary? In general, we strongly recommend using the binary on desktops and laptops. If you are wanting to do development, then we recommend using the development Docker container and the current main (development) branch; instructions for that are posted at PyLith development environment — PyLith Installer v3.0.0-0dev documentation.
Ok, thank you. Compiling with --enable-openssl works and does not give error. The problem appears later with numpy… pytia.
We are interested in compiling and testing a friction law that is not part of the binary distribution.
Ok, thanks! I will follow the instructions “posted at PyLith development environment”.
If you want to add an alternative friction model, then you are currently stuck with v2.2.2 as we have not yet re-implemented friction in the PyLith main branch that has multiphysics capabilities.
I know the error you are getting is with numpy, but the module it is having trouble with is associated with OpenSSL. In particular, it happens when the hashlib module, which is a standard Python distribution, cannot be built when installing Python, because it can’t find OpenSSL or has trouble using it. In order to provide additional assistance, please provide the config.log from running the PyLith installer configure and the config.log from the installation of Python.
This site does not let me attach config.log (it says I am a new user, I do not understand).
I have continued with my compilation, I have advanced but I still have errors. After compiling netcdf it passes numpy, then to Cython and here I have the error with h5py:
writing build/bdist.macosx-10.13-intel/egg/EGG-INFO/native_libs.txt
creating dist
creating ‘dist/Cython-0.29.16-py2.7-macosx-10.13-intel.egg’ and adding ‘build/bdist.macosx-10.13-intel/egg’ to it
removing ‘build/bdist.macosx-10.13-intel/egg’ (and everything under it)
Processing Cython-0.29.16-py2.7-macosx-10.13-intel.egg
creating /Users/cyclo/pylith_222_1/lib/python2.7/site-packages/Cython-0.29.16-py2.7-macosx-10.13-intel.egg
Extracting Cython-0.29.16-py2.7-macosx-10.13-intel.egg to /Users/cyclo/pylith_222_1/lib/python2.7/site-packages
Adding Cython 0.29.16 to easy-install.pth file
Installing cython script to /Users/cyclo/pylith_222_1/bin
Installing cygdb script to /Users/cyclo/pylith_222_1/bin
Installing cythonize script to /Users/cyclo/pylith_222_1/bin
Installed /Users/cyclo/pylith_222_1/lib/python2.7/site-packages/Cython-0.29.16-py2.7-macosx-10.13-intel.egg
Processing dependencies for Cython==0.29.16
Finished processing dependencies for Cython==0.29.16
/Users/cyclo/Pylith222_1/src/pylith/pylith-installer-2.2.2-1/bin/fetch.sh “/opt/local/bin/curl -L -O -o” h5py-2.9.0.tar.gz Index of /~buildbot/deps
Found local copy of h5py-2.9.0.tar.gz.
/usr/bin/tar -zxf h5py-2.9.0.tar.gz
cd h5py-2.9.0 &&
python setup.py configure --hdf5=/Users/cyclo/pylith_222_1 &&
CC=mpicc CPPFLAGS="-I/Users/cyclo/pylith_222_1/include -I/Users/cyclo/pylith_222_1/include " LDFLAGS="-L/Users/cyclo/pylith_222_1/lib -L/Users/cyclo/pylith_222_1/lib64 -L/Users/cyclo/pylith_222_1/lib -L/Users/cyclo/pylith_222_1/lib64 " CFLAGS="-g -O2" CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS="-g -O2" python setup.py build &&
python setup.py install --prefix=/Users/cyclo/pylith_222_1 &&
touch /Users/cyclo/Pylith222_1/installed_h5py
Couldn’t find index page for ‘pkgconfig’ (maybe misspelled?)
No local packages or download links found for pkgconfig
Traceback (most recent call last):
File “setup.py”, line 168, in
cmdclass = CMDCLASS,
File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py”, line 111, in setup
_setup_distribution = dist = klass(attrs)
File “/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py”, line 268, in init
self.fetch_build_eggs(attrs[‘setup_requires’])
File “/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py”, line 313, in fetch_build_eggs
replace_conflicting=True,
File “/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/init.py”, line 843, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File “/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/init.py”, line 1088, in best_match
return self.obtain(req, installer)
File “/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/init.py”, line 1100, in obtain
return installer(requirement)
File “/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py”, line 380, in fetch_build_egg
return cmd.easy_install(req)
File “/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/command/easy_install.py”, line 632, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse(‘pkgconfig’)
make[2]: *** [h5py] Error 1
make[1]: *** [installed_h5py] Error 2
make: *** [installed_pylith] Error 2
What am I missing?
You may need to rename the file in order to attach it, for example to something like config_log.txt. If you can’t attach the file, then post your configure arguments to the PyLith installer.
Note: I only have access to a mac with Big Sur. I tried to replicate your error but I end up with some different problems associated with incompatibilities with Big Sur and older versions of packages like OpenSSL and HDF5.
When I leave --disable-h5py , the compilation proceeds but with a new error. I could not attached config.log. Please can you check the shared link?
I am working on updating the PyLith installer to resolve several issues building v2.2.2. I am currently testing the updated version and updating the documentation. I hope to have a version you can try in the next day or so.
Excellent, thank you very much, I look forward to the new publication.
I have released version 2.2.2-2 of the PyLith Installer. See https://community.geodynamics.org/t/pylith-installer-v2-2-2-2-now-available.
Hello, have been trying to compile the latest version from Installer 2.2.2-2. but I still have errors. I used macOS High Sierra, Catalina and BigSur. I had better results on Catalina.
I share the logs in the following link (test-suite.log resulted in Catalina 10.15 ). I hope you can help me.
Googling the error you get wit Big Sur (configure: error: Unexpected output of ‘arch’ on OSX), it looks like Python 2.7 is not setup to work with the M1 Processor.
The failure to build automake on High Sierra baffles me. The only suggestion I have is to remove all non-system directories, such as /Users/marcont/miniconda3/* and /opt/local/*, from your PATH.
The failure with running the spatialdata/geocoords test on Catalina may be related to incompatibilities with other software. I suggest removing all of the non-system directories from your PATH.
You may have better luck with using the Docker container for PyLith development. All of the dependencies are provided and the environment is all setup so you don’t have to worry about conflicts with other software. VS Code (and some other IDEs) allows seamless access to the container, so it “feels” like you are working with local files.
Hi, good news on build 2.2.2.-2 on Catalina 10.15, it shows no errors and passed all tests. As you indicated miniconda3 was interfering.
I will check later how to use pylith in Docker for now I have no experience in using Docker. Thank you very much for the assistance.