conda pkg openssl is restricted to version <3.1 - this prevents installation of recent r-base

[schlichtanders]

Not sure whether this is a bug, probably not?

How can we rely on more recent openssl version, e.g. in order to install a recent r-base next to PythonCall?

I got the following error

error    libmamba Could not solve for environment specs
    The following packages are incompatible
    ├─ openssl >=3,<3.1  is requested and can be installed;
    └─ r 4.4  is not installable because it requires
       └─ r-base >=4.4,<4.5.0a0  but there are no viable options
          ├─ r-base [4.4.0|4.4.1] would require
          │  └─ libcurl >=8.8.0,<9.0a0  but there are no viable options
          │     ├─ libcurl [8.10.0|8.10.1] would require
          │     │  └─ openssl >=3.3.2,<4.0a0 , which conflicts with any installable versions previously reported;
          │     ├─ libcurl 8.11.1 would require
          │     │  └─ openssl >=3.4.0,<4.0a0 , which conflicts with any installable versions previously reported;
          │     ├─ libcurl 8.8.0 would require
          │     │  └─ openssl >=3.3.0,<4.0a0 , which conflicts with any installable versions previously reported;
          │     └─ libcurl [8.8.0|8.9.0|8.9.1] would require
          │        └─ openssl >=3.3.1,<4.0a0 , which conflicts with any installable versions previously reported;
          ├─ r-base 4.4.1 would require
          │  └─ libcurl [>=8.9.0,<9.0a0 |>=8.9.1,<9.0a0 ], which cannot be installed (as previously explained);
          ├─ r-base [4.4.1|4.4.2] would require
          │  └─ libcurl >=8.10.1,<9.0a0 , which cannot be installed (as previously explained);
          └─ r-base 4.4.2 would require
             └─ libcurl >=8.11.1,<9.0a0 , which cannot be installed (as previously explained).

[cjdoris]

Yep unfortunately Julia currently uses OpenSSL 3.0 (this is apparently intentional) so for compatibility reasons we restrict the version of OpenSSL installed by CondaPkg.

You can override this by setting the openssl_version preference to ignore. However YMMV and you'll likely get errors in any code that actually uses OpenSSL.

This is an annoying incompatibility - you can't install new versions of Python for the same reason. I need to open a dialogue with Julia maintainers about this.

[schlichtanders]

Thank you so much for your comment. For Python it is the same? Indeed a big pity...
Also thanks for pointing to the preference option.

Please feel free to close this issue or keep it open, whatever you like.

[wilf-phasecraft]

I believe Julia now uses a more up to date version of OpenSSL, which is now causing problems when installing certain packages with JuliaCall/PythonCall.

In short, the following error is commonly encountered (on Linux at least)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wilf/.julia/packages/PythonCall/WMWY0/src/JlWrap/module.jl", line 27, in seval
    return self._jl_callmethod($(pyjl_methodnum(pyjlmodule_seval)), expr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juliacall.JuliaError: InitError: could not load library "/home/wilf/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so"
/lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by /home/wilf/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/GfYNv/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ OpenSSL_jll ~/.julia/packages/OpenSSL_jll/WaTt1/src/wrappers/x86_64-linux-gnu.jl:16
  [5] run_module_init(mod::Module, i::Int64)
    @ Base ./loading.jl:1134
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1122
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1067
  [8] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1581
  [9] _tryrequire_from_serialized(modkey::Base.PkgId, build_id::UInt128)
    @ Base ./loading.jl:1445
 [10] _tryrequire_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String)
    @ Base ./loading.jl:1524
 [11] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1990
 [12] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [13] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [14] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [15] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [16] macro expansion
    @ ./loading.jl:1790 [inlined]
 [17] macro expansion
    @ ./lock.jl:267 [inlined]
 [18] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [19] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [20] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [21] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
 [22] eval
    @ ./boot.jl:385 [inlined]
 [23] eval
    @ ./Base.jl:88 [inlined]
 [24] pyjlmodule_seval(self::Module, expr::Py)
    @ PythonCall.JlWrap ~/.julia/packages/PythonCall/WMWY0/src/JlWrap/module.jl:13
 [25] _pyjl_callmethod(f::Any, self_::Ptr{PythonCall.C.PyObject}, args_::Ptr{PythonCall.C.PyObject}, nargs::Int64)
    @ PythonCall.JlWrap ~/.julia/packages/PythonCall/WMWY0/src/JlWrap/base.jl:67
 [26] _pyjl_callmethod(o::Ptr{PythonCall.C.PyObject}, args::Ptr{PythonCall.C.PyObject})
    @ PythonCall.JlWrap.Cjl ~/.julia/packages/PythonCall/WMWY0/src/JlWrap/C.jl:63
during initialization of module OpenSSL_jll

To reproduce:

1. Create a new python venv using python3.12 -m venv .venv
2. pip install juliacall (by default this installs Julia 1.11.5)
3. Start python and run

>>> from juliacall import Main as jl
>>> from juliapkg import add, resolve
>>> add("Yao", "5872b779-8223-5990-8dd0-5abbb0748c8c") # other packages with OpenSSL deps also cause the error
>>> resolve()
>>> jl.seval("using Yao")

Info:
Python: 3.12.10
Julia: 1.11.5
OS: Linux

[rconnorjohnstone]

I started running into this problem yesterday too. I'm in need of a quick solution, so I'm hoping downgrading julia within my docker container will fix things for now