Differences

This shows you the differences between two versions of the page.

--- basic_users_guide_xces_new [2025/02/12 09:39] – [Using XCES/Freva R/Python environment to locally install your own packages] etorma
+++ basic_users_guide_xces_new [2025/05/19 11:50] (current) – [Installing the freva library] etorma
@@ Line 1: / Line 1: @@
 ===== Getting familiar with the XCES Platform ======
+<WRAP round info>
+  * For a **quick tour** through Freva and Freva Plugins please [[https://freva-clint.github.io/Talks/Freva_Workshop_2024/index.slides.html|take a look here]].
+  * You also have some **hands-on exercises** to get yourself familiarised with freva and plugins [[https://gitlab.dkrz.de/freva/freva_workshop|here]].
+</WRAP>
 The XCES platform is an instance of the Free Evaluation System Framework (Freva), designed by the Freva team and adapted for the scientists of the ClimXtreme project by Module D. Below you will find relevant information on how to get started with the system.
-The system can used either through the [[basic_users_guide_xces_new#Access through the shell|shell]], [[basic_users_guide_xces_new#Access through the web|website]], or [[basic_users_guide_xces_new#access_through_python_module_pypi_jupyter|its python module (e.g. for Jupyter)]]. We will give an overview of the capabilities of XCES on all platforms and also the necessary steps to set a working environment in the shell that will allow us [[https://freva-clint.github.io/freva/FrevaCli.html#add-new-data-to-the-databrowser|to share data]] with the rest of the ClimXtreme community or test our [[list_of_plugins|plugins]] in development stage.
+The system can used either through the [[basic_users_guide_xces_new#Access through the shell|shell]], [[basic_users_guide_xces_new#Access through the web|website]], or [[basic_users_guide_xces_new#access_through_python_module_pypi_jupyter|its python module (e.g. for Jupyter)]]. We will give an overview of the capabilities of XCES on all platforms and also the necessary steps to set a working environment in the shell that will allow us [[https://freva-clint.github.io/freva/FrevaCli.html#add-new-data-to-the-databrowser|to share data]] with the rest of the ClimXtreme community or test our [[https://gitlab.dkrz.de/bm1159/plugins4freva/gitlab-profile/-/blob/main/README.md|plugins]] in development stage.
@@ Line 189: / Line 195: @@
 BUT be aware that there is currently NO API to call freva functionalities within R, so it will only work as an additional R kernel, nothing more.
-===== Creating your own conda/mamba environment with the freva library =====
+===== Creating your own conda/mamba environment with the freva library (e.g. for plugins) =====
-If, on the other hand you want to use freva as a python library alongside others (e.g. in a conda environment) then you can:
+We **strongly** suggest creating your own environment to run your plugin code. There are several reasons for that, among them:
-  - install the package via pypi ''pip install freva'' or via conda/mamba ''mamba install -c conda-forge freva''
+  - to not rely on centrally offered environments, making the plugins easier to mantain and port other instituions / newer HPCs (e.g. the successor of Levante)
-  - add the environment config of XCES in your script, e.g.:
+  - to offer flexibility (e.g. via Makefile) to install other packages / binaries via other means (e.g. directly CRAN, other means)
-<code python>
+  - to allow more reproducibile environment (ideally if e..g the library versions are set frozen).
-import os,sys
-os.environ["EVALUATION_SYSTEM_CONFIG_FILE"] = "/work/bm1159/XCES/freva/evaluation_system.conf" # this is for the environment
+A way of doing it is creating a ''mamba/conda'' environment, you can use the conda/mamba installer provided by loading freva module for that (''module load clint xces'')), e.g.:
-os.environ["EVALUATION_SYSTEM_CONFIG_DIR"] = "/work/bm1159/XCES/freva" # this is for the environment
+  * Create a New Environment in Default Location (i.e. '~/.conda/envs/' be aware that your home only has 25GB!):
-import freva
+<code>
+conda create --name myenv
 </code>
+to find your conda env you simply do ''conda env list'' and ''conda activate myenv'' (you may need to add something to your .bashrc/.zshrc)
+  * Create a New Environment in a Specific Folder (e.g. in a plugin repo):
+<code>
+conda create --prefix /path/to/myenv
+</code>
+  * Create a New Environment with specific libraries and e.g. python version:
+<code>
+conda create --name myenv python=3.8 numpy pandas
+</code>
+You can do more elaborate installations for e.g. your plugin environments, please [[https://freva-clint.github.io/freva/developers_guide.html|take a look at this short guide for it]].
 <WRAP round important>
-**NOTE**: since 2024 DKRZ does not allow access to ''default'' channels in conda [[https://legal.anaconda.com/policies/en/#terms-of-service| for reasons]]. You need to make sure you will not use it or you will get an ''HTTP error'' message while trying to create an env. Options:
+**NOTE**: since 2024 DKRZ does not allow access to ''default'' channels in conda [[https://legal.anaconda.com/policies/en/#terms-of-service| due to changes in Anaconda's terms of services]]. You need to make sure you will not use it or you will get an ''HTTP error'' message while trying to create an env. Options:
   * use flag '' --override-channels'' so it does not search default.
-  * create ''$HOME.condarc'' file with:
+  * create ''$HOME/.condarc'' file with:
 <code>
 channels:
@@ Line 224: / Line 242: @@
-===== Using XCES/Freva R/Python environment to locally install your own packages =====
+==== Creating a jupyter kernel out of your environment ====
+For that, you will need to previously install the ''ipykernel'' library via e.g.:
+<code python>
+conda create --name myenv python=3.8 ipykernel # with conda
+python -m pip install ipykernel # with pip
+</code>
+And then, given you are using the python binary of your environment:
+<code python>
+python -m ipykernel install --user --name=$(KERNEL_NAME) --display-name $(KERNEL_NAME)
+</code>
+==== Installing the freva library ====
+If you want to use freva as a python library alongside others (e.g. in a conda environment) then you can:
+  - install the package via pypi ''pip install freva'' or via conda/mamba ''mamba install -c conda-forge freva''
+  - add the environment config of XCES in your script, e.g.:
+<code python>
+import freva
+freva.config(config_file="/work/bm1159/XCES/freva/evaluation_system.conf")
+</code>
+that should work on a python script, python env console and jupyter notebook (e.g. [[https://jupyterhub.dkrz.de|DKRZ's jupyterhub]].
+Incidentally, you can load plugins using the same command, e.g.:
+<code python>
+import freva
+freva.config(config_file="/path/to/the/freva/config/file.conf",
+             plugin_path="~/freva/mypluing,the_plugin")
+</code>
+===== Using XCES/Freva R/Python environment to locally install your own packages [NOT RECOMMENDED] =====
 <WRAP round info>