{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "5fbc1dee",
      "metadata": {
        "id": "5fbc1dee"
      },
      "source": [
        "# July 2021 floods: starter notebook\n",
        "\n",
        "This notebook does two things:\n",
        "1. **Connects** you to the challenge data in the cloud. Nothing is downloaded, and no login is needed: the data is public.\n",
        "2. **Shows what data is available** (EMS flood maps, EFAS flood hazard, GHS-OBAT buildings, GHS-POP population for DE, BE, NL).\n",
        "\n",
        "Click **Run all**. Works in Colab and on a laptop.\n",
        "\n",
        "<details>\n",
        "<summary><b>Not using Colab? (VS Code, Jupyter, any Python)</b></summary>\n",
        "\n",
        "1. `pip install gcsfs pyogrio rasterio geopandas shapely ipykernel`\n",
        "   (on Windows, if that fails: `conda install -c conda-forge geopandas rasterio pyogrio gcsfs`)\n",
        "2. VS Code: install the Python and Jupyter extensions and pick your environment as kernel. If the packages are already installed there, you can skip the `pip install` cell below. After installing, restart the kernel.\n",
        "\n",
        "**Problems:** *connection or timeout errors* → check your internet connection and retry · *`No such file` on a path* → run the Explore cell to see the exact file names · *`ModuleNotFoundError`* → the kernel and your terminal use different Pythons: install with `%pip install gcsfs` inside the notebook, then restart the kernel.\n",
        "\n",
        "</details>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1fa4f5c8",
      "metadata": {
        "id": "1fa4f5c8"
      },
      "source": [
        "## 1. Setup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "id": "2f1a568b",
      "metadata": {
        "id": "2f1a568b"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "%pip install -q gcsfs pyogrio rasterio geopandas shapely"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "3a6951de",
      "metadata": {
        "id": "3a6951de"
      },
      "outputs": [],
      "source": [
        "import os, re, warnings\n",
        "from pathlib import Path\n",
        "import pandas as pd\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "\n",
        "BUCKET = \"datasets_challenge\"\n",
        "FOLDER = \"working_data\""
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7e4db53a",
      "metadata": {
        "id": "7e4db53a"
      },
      "source": [
        "## 2. Connect"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "96bc9f92",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "96bc9f92",
        "outputId": "54d6c138-933d-488e-9848-91d90dd95bbd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Connected: ['cems-efas', 'ems_be', 'ems_de', 'ems_nl', 'ghs_pop', 'obat_be', 'obat_de', 'obat_nl']\n"
          ]
        }
      ],
      "source": [
        "import gcsfs\n",
        "\n",
        "# The bucket is public: no login needed.\n",
        "os.environ[\"GS_NO_SIGN_REQUEST\"] = \"YES\"      # lets GDAL (rasterio, pyogrio) read it anonymously\n",
        "GDAL_PREFIX = \"/vsigs/\"\n",
        "ROOT = f\"{BUCKET}/{FOLDER}\"\n",
        "fs = gcsfs.GCSFileSystem(token=\"anon\")\n",
        "print(\"Connected:\", [Path(p).name for p in fs.ls(ROOT)])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ce75ad49",
      "metadata": {
        "id": "ce75ad49"
      },
      "source": [
        "## 3. Explore"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "3b82f013",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "3b82f013",
        "outputId": "2bd3a22b-f62c-4c53-ed05-cc737b3377aa"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "cems-efas\n",
            "   Europe_RP100_filled_depth.tif 323 MB\n",
            "   Europe_RP10_filled_depth.tif 272 MB\n",
            "   Europe_RP500_filled_depth.tif 350 MB\n",
            "\n",
            "ghs_pop\n",
            "   GHS_POP_E2020_GLOBE_R2023A_4326_30ss_V1_0.tif 383 MB\n",
            "   GHS_POP_E2025_GLOBE_R2023A_4326_30ss_V1_0.tif 384 MB\n",
            "\n",
            "obat_be\n",
            "   GHS_OBAT_GPKG_BEL_E2020_R2024A_V1_0.gpkg 1,433 MB\n",
            "\n",
            "obat_de\n",
            "   GHS_OBAT_GPKG_DEU_E2020_R2024A_V1_0.gpkg 8,579 MB\n",
            "\n",
            "obat_nl\n",
            "   GHS_OBAT_GPKG_NLD_E2020_R2024A_V1_0.gpkg 2,229 MB\n",
            "\n",
            "ems_be\n",
            "   EMSR518_AOI01_DEL_MONIT01_r1_VECTORS_v1_vector 9 shp, 65.4 MB\n",
            "   EMSR518_AOI01_DEL_PRODUCT_r1_VECTORS_v1_vector 10 shp, 76.3 MB\n",
            "   EMSR518_AOI02_DEL_MONIT01_r1_VECTORS_v1_vector 12 shp, 110.3 MB\n",
            "   EMSR518_AOI02_DEL_PRODUCT_r1_VECTORS_v1_vector 10 shp, 138.3 MB\n",
            "\n",
            "ems_de\n",
            "   EMSR517_AOI01_DEL_MONIT01_r1_RTP01_v1_vector 11 shp, 21.3 MB\n",
            "   EMSR517_AOI01_DEL_PRODUCT_r1_RTP01_v1_vector 10 shp, 21.3 MB\n",
            "   EMSR517_AOI03_DEL_MONIT01_r1_RTP01_v1_vector 10 shp, 6.9 MB\n",
            "   EMSR517_AOI03_DEL_PRODUCT_r1_RTP01_v1_vector 10 shp, 5.2 MB\n",
            "   EMSR517_AOI04_DEL_MONIT01_r1_RTP01_v1_vector 11 shp, 22.4 MB\n",
            "   EMSR517_AOI04_DEL_PRODUCT_r1_RTP01_v1_vector 10 shp, 26.6 MB\n",
            "   EMSR517_AOI05_DEL_MONIT01_r1_RTP01_v1_vector 10 shp, 102.6 MB\n",
            "   EMSR517_AOI05_DEL_PRODUCT_r1_RTP01_v1_vector 10 shp, 69.2 MB\n",
            "   EMSR517_AOI06_DEL_MONIT01_r1_RTP01_v1_vector 10 shp, 78.7 MB\n",
            "   EMSR517_AOI06_DEL_PRODUCT_r1_RTP01_v1_vector 10 shp, 77.2 MB\n",
            "   EMSR517_AOI07_GRA_PRODUCT_r1_RTP01_v1_vector 9 shp, 6.6 MB\n",
            "   EMSR517_AOI08_GRA_PRODUCT_r1_RTP01_v1_vector 9 shp, 5.5 MB\n",
            "   EMSR517_AOI09_GRA_PRODUCT_r1_RTP01_v1_vector 10 shp, 11.8 MB\n",
            "   EMSR517_AOI10_GRA_PRODUCT_r1_RTP01_v1_vector 10 shp, 4.3 MB\n",
            "   EMSR517_AOI11_GRA_PRODUCT_r1_RTP01_v1_vector 10 shp, 7.6 MB\n",
            "   EMSR517_AOI14_GRA_PRODUCT_r1_RTP01_v1_vector 9 shp, 13.4 MB\n",
            "   EMSR517_AOI15_GRA_MONIT01_r1_RTP01_v3_vector 9 shp, 25.6 MB\n",
            "   EMSR517_AOI15_GRA_PRODUCT_r1_RTP01_v1_vector 8 shp, 12.9 MB\n",
            "   EMSR517_AOI16_GRA_PRODUCT_r1_RTP01_v1_vector 9 shp, 4.3 MB\n",
            "   EMSR517_AOI17_GRA_PRODUCT_r1_RTP01_v2_vector 9 shp, 7.3 MB\n",
            "   EMSR517_AOI18_GRA_PRODUCT_r1_RTP01_v1_vector 7 shp, 2.9 MB\n",
            "   EMSR517_AOI20_GRA_PRODUCT_r1_RTP01_v1_vector 8 shp, 2.1 MB\n",
            "\n",
            "ems_nl\n",
            "   EMSR520_AOI01_DEL_PRODUCT_r1_RTP01_v1_vector 10 shp, 59.8 MB\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "                                                                    layer\n",
              "cc aoi   ptype ver                                                       \n",
              "be AOI01 DEL   MONIT01  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "               PRODUCT  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "   AOI02 DEL   MONIT01  areaOfInterestA, builtUpA, facilitiesA, facili...\n",
              "               PRODUCT  areaOfInterestA, facilitiesL, hydrographyA, hy...\n",
              "de AOI01 DEL   MONIT01  areaOfInterestA, builtUpA, facilitiesA, facili...\n",
              "               PRODUCT  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "   AOI03 DEL   MONIT01  areaOfInterestA, builtUpA, facilitiesA, facili...\n",
              "               PRODUCT  areaOfInterestA, builtUpA, facilitiesA, facili...\n",
              "   AOI04 DEL   MONIT01  areaOfInterestA, builtUpA, facilitiesA, facili...\n",
              "               PRODUCT  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "   AOI05 DEL   MONIT01  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "               PRODUCT  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "   AOI06 DEL   MONIT01  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "               PRODUCT  areaOfInterestA, builtUpA, facilitiesL, hydrog...\n",
              "   AOI07 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI08 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI09 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI10 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, facili...\n",
              "   AOI11 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI14 GRA   PRODUCT  areaOfInterestA, builtUpP, hydrographyA, hydro...\n",
              "   AOI15 GRA   MONIT01  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "               PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI16 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI17 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI18 GRA   PRODUCT  areaOfInterestA, builtUpP, facilitiesA, hydrog...\n",
              "   AOI20 GRA   PRODUCT  areaOfInterestA, facilitiesA, hydrographyA, hy...\n",
              "nl AOI01 DEL   PRODUCT  areaOfInterestA, builtUpA, facilitiesA, hydrog..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e5288fce-be8a-4a96-90cb-77edf772a66f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>layer</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>cc</th>\n",
              "      <th>aoi</th>\n",
              "      <th>ptype</th>\n",
              "      <th>ver</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"4\" valign=\"top\">be</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI01</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">DEL</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI02</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">DEL</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesA, facili...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, facilitiesL, hydrographyA, hy...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"22\" valign=\"top\">de</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI01</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">DEL</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesA, facili...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI03</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">DEL</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesA, facili...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesA, facili...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI04</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">DEL</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesA, facili...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI05</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">DEL</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI06</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">DEL</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesL, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI07</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI08</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI09</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI10</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, facili...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI11</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI14</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, hydrographyA, hydro...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">AOI15</th>\n",
              "      <th rowspan=\"2\" valign=\"top\">GRA</th>\n",
              "      <th>MONIT01</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI16</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI17</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI18</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpP, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AOI20</th>\n",
              "      <th>GRA</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, facilitiesA, hydrographyA, hy...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>nl</th>\n",
              "      <th>AOI01</th>\n",
              "      <th>DEL</th>\n",
              "      <th>PRODUCT</th>\n",
              "      <td>areaOfInterestA, builtUpA, facilitiesA, hydrog...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e5288fce-be8a-4a96-90cb-77edf772a66f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e5288fce-be8a-4a96-90cb-77edf772a66f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e5288fce-be8a-4a96-90cb-77edf772a66f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"display(pd\",\n  \"rows\": 27,\n  \"fields\": [\n    {\n      \"column\": \"layer\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 17,\n        \"samples\": [\n          \"areaOfInterestA, builtUpA, facilitiesL, hydrographyA, hydrographyL, imageFootprintA, naturalLandUseA, observedEventA, transportationL\",\n          \"areaOfInterestA, builtUpA, facilitiesL, hydrographyA, hydrographyL, imageFootprintA, naturalLandUseA, observedEventA, physiographyL, transportationL\",\n          \"areaOfInterestA, builtUpA, facilitiesL, hydrographyA, hydrographyL, imageFootprintA, naturalLandUseA, observedEventA, transportationL, transportationP\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "for folder in [\"cems-efas\", \"ghs_pop\", \"obat_be\", \"obat_de\", \"obat_nl\", \"ems_be\", \"ems_de\", \"ems_nl\"]:\n",
        "    print(f\"\\n{folder}\")\n",
        "    files = fs.find(f\"{ROOT}/{folder}\", detail=True)\n",
        "    if folder.startswith(\"ems_\"):\n",
        "        # vector products: one line per product directory\n",
        "        dirs = {}\n",
        "        for p, info in sorted(files.items()):\n",
        "            d = dirs.setdefault(Path(p).parent.name, [0, 0])\n",
        "            d[0] += p.lower().endswith(\".shp\"); d[1] += int(info[\"size\"])\n",
        "        for name, (n_shp, size) in dirs.items(): print(\"  \", name, f\"{n_shp} shp, {size/1e6:,.1f} MB\")\n",
        "    else:\n",
        "        for p, info in sorted(files.items()):\n",
        "            if p.lower().endswith((\".tif\", \".tiff\", \".gpkg\")): print(\"  \", Path(p).name, f\"{int(info['size'])/1e6:,.0f} MB\")\n",
        "\n",
        "# EMS: one row per area (aoi), type (DEL/GRA) and version, with its layers\n",
        "EMS_RE = re.compile(r\"(EMSR\\d+)_(AOI\\d+)_([A-Z]+)_(PRODUCT|MONIT\\d*)_(\\w+?)_r\\d+_v\\d+\", re.I)\n",
        "rows = []\n",
        "for cc in (\"de\", \"be\", \"nl\"):\n",
        "    for p in fs.find(f\"{ROOT}/ems_{cc}\"):\n",
        "        m = EMS_RE.search(Path(p).name) if p.lower().endswith(\".shp\") else None\n",
        "        if m: rows.append(dict(cc=cc, aoi=m[2], ptype=m[3].upper(), ver=m[4].upper(), layer=m[5]))\n",
        "display(pd.DataFrame(rows).groupby([\"cc\", \"aoi\", \"ptype\", \"ver\"]).layer.apply(lambda s: \", \".join(sorted(s))).to_frame())"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c726ab00",
      "metadata": {
        "id": "c726ab00"
      },
      "source": [
        "## 4. Your analysis\n",
        "Read data straight from the bucket, no login needed: list files with `fs` (gcsfs), or open them with GDAL-based libraries using the `GDAL_PREFIX` path prefix, for example `rasterio.open(f\"{GDAL_PREFIX}{path}\")`. The bucket is read-only, so save results to the runtime disk and download them from the file browser."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "d08bb833",
      "metadata": {
        "id": "d08bb833"
      },
      "outputs": [],
      "source": [
        "# your code"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}