Topic: “requested datatype primary not available” when adding Remi repos

Hello everyone,

This question was originally posted at SuperUser: https://superuser.com/questions/1495051 … positories

But since nobody can answer there, I am going to repeat it here smile

In our daily autotests we install PHP 7.3 from Remi's repository using Ansible. Usually it works fine, but sometimes (few times a month) it fails with the error like below:


fatal: [vm1.int.zone]: FAILED! => {
"changed": false,
"invocation": {
    "module_args": {
        "allow_downgrade": false,
        "conf_file": null,
        "disable_gpg_check": false,
        "disable_plugin": [],
        "disablerepo": null,
        "enable_plugin": [],
        "enablerepo": "epel,remi,remi-php73",
        "exclude": null,
        "install_repoquery": true,
        "installroot": "/",
        "list": null,
        "name": [
            "php",
            "php-mcrypt",
            "php-mbstring",
            "php-xml",
            "php-xsl",
            "php-xmlrpc",
            "php-pdo",
            "php-cli",
            "php-pgsql",
            "php-pspell",
            "php-pecl-ssh2",
            "php-pecl-zip",
            "php-smbclient",
            "php-intl",
            "php-mysql",
            "php-curl",
            "php-xdebug",
            "php-process"
        ],
        "security": false,
        "skip_broken": false,
        "state": "installed",
        "update_cache": false,
        "update_only": false,
        "validate_certs": true
    }
},
"msg": "Failure talking to yum: requested datatype primary not available"


There are 3 Remi repos:

# ll -d /var/cache/yum/x86_64/7/remi*
drwxr-xr-x 4 root root 4096 Oct 22 22:12 /var/cache/yum/x86_64/7/remi
drwxr-xr-x 4 root root 4096 Oct 22 22:12 /var/cache/yum/x86_64/7/remi-php73
drwxr-xr-x 4 root root 4096 Oct 22 22:12 /var/cache/yum/x86_64/7/remi-safe



and any of these cause the problem if running certain yum commands, for example:

# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror2.int.zone
* epel: mirror2.int.zone
* extras: mirror2.int.zone
* remi-safe: mirror.serverion.com
* updates: mirror2.int.zone
Error: requested datatype primary not available


If disable Remi's repos, everything goes fine. The problem is in broken cache for Remi's repository, for example:

# ll /var/cache/yum/x86_64/7/remi-php73/
total 20
-rw-r--r-- 1 root root    0 Oct 22 22:12 cachecookie
drwxr-xr-x 2 root root 4096 Oct 22 22:12 gen
-rw-r--r-- 1 root root 2996 Oct 22 22:12 mirrorlist.txt
drwxr-xr-x 2 root root 4096 Oct 22 22:12 packages
-rw-r--r-- 1 root root 2259 Oct 22 22:12 repomd.xml

# ll /var/cache/yum/x86_64/7/remi-php73/gen/
total 0


Usually, this directory looks like this:

# ll /var/cache/yum/x86_64/7/remi-php73/
total 232
-rw-r--r-- 1 root root 211219 Oct 22 14:03 7042a64df80e22496ef9a31990242357a84898e0f0272915cf7cce20b705d863-primary.sqlite.bz2
-rw-r--r-- 1 root root      0 Oct 23 04:24 cachecookie
drwxr-xr-x 2 root root   4096 Oct 23 04:24 gen
-rw-r--r-- 1 root root   2996 Oct 23 04:24 mirrorlist.txt
drwxr-xr-x 2 root root   4096 Oct 23 04:25 packages
-rw-r--r-- 1 root root   3095 Oct 22 14:03 repomd.xml

# ll /var/cache/yum/x86_64/7/remi-php73/gen/
total 996
-rw-r--r-- 1 root root 1015808 Oct 22 14:03 primary_db.sqlite


So the files {long_id}-primary.sqlite.bz2 and gen/primary_db.sqlite are missing for some reason sometimes.
yum makecache recreates the cache and fixes the issue.

The question is what can be a reason of this?
Thank you!

Re: “requested datatype primary not available” when adding Remi repos

Looks like you have a cache issue, using a outdated repomd file.

As the repository is some time very active, with multiple update on the same day, these file change a lot.

BTW, to reduce possible bad effects, I already keep 8 generations of these files online

So a simple "yum clean metadata" should fix it

You pasted output is quite old, can you get a more recent one, to see how old is repomd when the issue happens ?
If the test script run once a day... this should not happen...

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: “requested datatype primary not available” when adding Remi repos

Good day Remi!

Thank you for such a prompt reply.
Today the issue stroke again, so I have a fresh example:

[root@Container-ecddd072dba4 ~]# date
Fri Nov 22 10:16:02 MSK 2019

[root@Container-ecddd072dba4 ~]# tail -2 /var/log/yum.log
Nov 21 18:12:40 Installed: epel-release-7-12.noarch
Nov 21 18:12:40 Installed: remi-release-7.7-1.el7.remi.noarch

[root@Container-ecddd072dba4 ~]# ll /var/cache/yum/x86_64/7/remi-php73/
total 20
-rw-r--r-- 1 root root    0 Nov 21 18:12 cachecookie
drwxr-xr-x 2 root root 4096 Nov 21 18:12 gen
-rw-r--r-- 1 root root 3056 Nov 21 18:12 mirrorlist.txt
drwxr-xr-x 2 root root 4096 Nov 21 18:12 packages
-rw-r--r-- 1 root root 2259 Nov 21 18:12 repomd.xml

[root@Container-ecddd072dba4 ~]# ll /var/cache/yum/x86_64/7/remi-php73/gen/
total 0

[root@Container-ecddd072dba4 ~]# cat /var/cache/yum/x86_64/7/remi-php73/repomd.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "XXXXX"><html xmlns=" www.w3.org/1999/xhtml"><head><meta name="viewport" content="width=device-width, initial-scale=1"/><title>mirror.cedia.org.ec : /remi/enterprise/7/php73/x86_64/repodata/repomd.xml</title><link rel="shortcut icon" type="image/vnd.microsoft.icon" href="/favicon.ico"/><link href="/mirror.css" rel="stylesheet" type="text/css"/></head><body><div class="header"><div class="stitle">Mirror OSS CEDIA</div></div><h1>Index of /remi/enterprise/7/php73/x86_64/repodata/repomd.xml</h1><div class="list"><table summary="Directory Listing" cellpadding="0" cellspacing="0"><thead><tr><th class="n">Name</th><th class="m">Last Modified</th><th class="s">Size</th><th class="t">Type</th></tr></thead><tbody><tr><td class="n"><a href="../">Parent Directory</a>/</td><td class="m"> </td><td class="s">-  </td><td class="t">Directory</td></tr>15743229859d478ec2dde1d1ca5a8e79546585bfc0cdc6f9a078d06e6eb18a9c51d87d174c6d996d616ce16d9029dab38773ca85d94927ac8d30f7506de0ddaff3ee5ed8b515743229859234910420318f414af3c7d453f6b65f69e84b469cd025f33a86d330d053e8aed2e229f06bd48ea4cbd5fbeb497875797e5c282020c9ebbc10725f26b8ce9214a1f7e45ac29215743229858540410750636f796c14a93a4576895601e472681d7960d413798c25f14fb51cbc782b19733a6999460647a27f2466147b08c8f58070b8bac24125ff8d22dc739e363ee230cb1574322985270412054329151b14a63e498b97ce8cbe9fe44db2b75abae434e22ff346b7922a9ddf7e301f1dd1b6491dfc9686d492f0fa18b497174a99f767f0fb097963ef9cfa863c35b15743229852126331028096100461b85d665c13a6aaddb2bb5f62a597be0e1b4c02da4cf61148b2a5b3d281df9dee838d1d39c9e5b18d0c3cd4e6bcc5922ad76fc42a780a175d40dfe302717f157432298511985359392010c339b69454679873211e02b2e0ea407f23187dd72e8e49a18241b7f7539bc4efa9af29bebc59d355b6521905e211999ee73c33b6942816474d1a94c7e78d2a7a15743229854039518841610</tbody></table></div><div class="powered foot">
            By/Contact:
            <a href="mailto:mirror@cedia.org.ec?Subject=CEDIA%20mirror%20question">
                Paul Bernal Barzallo &amp; Ernesto Pérez Estévez
            </a>
            © <a href="XXXXX" target="_blank">Red CEDIA</a> 2014-2019
        </div></body></html>
[root@Container-ecddd072dba4 ~]#



So, we have our test environment automatically deployed from scratch and configured daily. Right after a virtual environment is created, an Ansible playbook starts (just a snippet responsible for adding the repo):

...
  tasks:
    - name: Enable EPEL repo
      yum_repository:
        name: epel
        description: EPEL YUM repo
        mirrorlist: mirrorlist.ap.int.zone/epel/{{ ansible_distribution_major_version }}/$basearch
        enabled: yes
        repo_gpgcheck: no
        gpgcheck: no
...

    - name: Add repository 'remi-repo' for PHP7
      yum:
        name: rpms.famillecollet.com/enterprise/remi-release-7.rpm
        state: present
      when: php_version == 'php7'
     
    - name: Installing PHP7 packages
      yum:
        enablerepo: epel,remi,remi-php73
        state: installed
        name:
          - php
          - php-mcrypt
          - php-mbstring
          - php-xml
          - php-xsl
          - php-xmlrpc
          - php-pdo
          - php-cli
          - php-pgsql
          - php-pspell
          - php-pecl-ssh2
          - php-pecl-zip
          - php-smbclient
          - php-intl
          - php-mysql
          - php-curl
          - php-xdebug
          - php-process 
      when: php_version == 'php7'

(Sorry, the form does not allow to post more than 1 http link, so I removed HTTP:// from URLs)

So as you can see from the Ansible output and other logs, the repo is enabled, but further PHP packages installation fails due to the issue with the cache:


fatal: [Container-ecddd072dba4.aqa.int.zone]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "allow_downgrade": false,
            "conf_file": null,
            "disable_gpg_check": false,
            "disable_plugin": [],
            "disablerepo": null,
            "enable_plugin": [],
            "enablerepo": "epel,remi,remi-php73",
            "exclude": null,
            "install_repoquery": true,
            "installroot": "/",
            "list": null,
            "name": [
                "php",
                "php-mcrypt",
                "php-mbstring",
                "php-xml",
                "php-xsl",
                "php-xmlrpc",
                "php-pdo",
                "php-cli",
                "php-pgsql",
                "php-pspell",
                "php-pecl-ssh2",
                "php-pecl-zip",
                "php-smbclient",
                "php-intl",
                "php-mysql",
                "php-curl",
                "php-xdebug",
                "php-process"
            ],
            "security": false,
            "skip_broken": false,
            "state": "installed",
            "update_cache": false,
            "update_only": false,
            "validate_certs": true
        }
    },
    "msg": "Failure talking to yum: requested datatype primary not available"
}


So it does not seem to be an issue with outdated repomd.

Re: “requested datatype primary not available” when adding Remi repos

In this case, you are bitten by an broken mirror.

And indeed this was reported by the monitoring
so I just disabled it.

BTW, I slightly improve the metadata cleanup logic to keep 8 files AND all files younger than 1 day.

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: “requested datatype primary not available” when adding Remi repos

Hmm... sadly, monitoring is unable to report difference between "offline" mirror (not an issue, yum will use another one) and "corrupted" mirror, which raise your issue.

I need to improved the used script...

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: “requested datatype primary not available” when adding Remi repos

Thank a lot!

Re: “requested datatype primary not available” when adding Remi repos

> I need to improved the used script...

Done. I will now received different report for offline / corrupted mirrors

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi