1 Topic by jwage (edited by jwage 2016-07-07 15:39:51)

  • Registered: 2016-07-07
  • Posts: 11

Topic: PHP crashes in shutdown handler with php-pecl-mongodb

We are experiencing a PHP crash using the php-pecl-mongodb extension, version 1.1.7. We are trying to get a full backtrace with debug symbols for the developers of php-pecl-mongodb but we're having trouble getting the debug symbols installed. Am I understanding correctly that this package doesn't have debug symbols compiled in by default? Any ideas how we could proceed?

Also worth mentioning that this crash looks like the trace from https://jira.mongodb.org/browse/PHPC-671 which was fixed in 1.1.7

Does the 1.1.7 version from remi have patches in it that are not in the official pecl version?

Installed Packages
Name        : php-pecl-mongodb
Arch        : x86_64
Version     : 1.1.7
Release     : 2.el6.remi.5.5
Size        : 727 k
Repo        : installed
From repo   : remi-php55
Summary     : MongoDB driver for PHP
URL         :
License     : BSD
Description : The purpose of this driver is to provide exceptionally thin glue between
            : MongoDB and PHP, implementing only fundemental and performance-critical
            : components necessary to build a fully-functional MongoDB driver.
            :
            : Package built for PHP 5.5.

Here is the backtrace without all the debug symbols.

#0  0x00000033ce632625 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00000033ce633e05 in abort () at abort.c:92
#2  0x00007f48d50e2415 in ?? () from /usr/lib64/libmongoc-priv.so.0
#3  0x00007f48d50e2838 in mongoc_cluster_stream_for_server () from /usr/lib64/libmongoc-priv.so.0
#4  0x00007f48d50dd1a6 in _mongoc_client_kill_cursor () from /usr/lib64/libmongoc-priv.so.0
#5  0x00007f48d50e9028 in _mongoc_cursor_destroy () from /usr/lib64/libmongoc-priv.so.0
#6  0x00007f48d533a7a2 in php_phongo_cursor_free () from /usr/lib64/php/modules/mongodb.so
#7  0x00007f48d5331bd1 in ?? () from /usr/lib64/php/modules/mongodb.so
#8  0x00000000005deedc in zend_objects_store_free_object_storage ()
#9  0x00000000005a7d53 in ?? ()
#10 0x00000000005b5e32 in ?? ()
#11 0x0000000000555cac in php_request_shutdown ()
#12 0x000000000066ece9 in ?? ()
#13 0x00000033ce61ed5d in __libc_start_main (main=0x66dfb0, argc=2, ubp_av=0x7fff944c98b8, init=<value optimized out>, fini=<value optimized out>, rtld_fini=<value optimized out>, stack_end=0x7fff944c98a8) at libc-start.c:226
#14 0x00000000004210b9 in _start ()

  • Remi
  • Remi
  • Administrateur
  • Offline
  • From: Champagne...
  • Registered: 2009-04-28
  • Posts: 3,834

Re: PHP crashes in shutdown handler with php-pecl-mongodb

> Does the 1.1.7 version from remi have patches in it that are not in the official pecl version?
No.

BTW, latest version is 1.1.8
The backtrace with php-debuginfo, php-pecl-mongodb-debuginfo and mongo-c-driver-debuginfo will be more useful.

Can you reproduce with latest 1.1.8 ?
Is php-pecl-mongo also installed ? is yes, can you try with only one ?

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Remi's Website

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

I am having trouble installing mongo-c-driver-debuginfo. I get the following:

Could not find debuginfo for main pkg: mongo-c-driver-1.3.5-1.el6.remi.x86_64

I was able to install php-pecl-mongodb-debuginfo successfully though. Here is the updated trace:

(gdb) bt full
#0  0x0000003b20e32625 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
        resultvar = 0
        pid = <value optimized out>
        selftid = <value optimized out>
#1  0x0000003b20e33e05 in abort () at abort.c:92
        save_stage = 2
        act = {__sigaction_handler = {sa_handler = 0x127121d40f60, sa_sigaction = 0x127121d40f60}, sa_mask = {__val = {253958350976, 20227416, 20215312, 0, 140736598182256, 1, 253955669547, 206158430256,
              140736598180856, 140736598180640, 0, 12821671817868660429, 5625048504752721613, 0, 140011908422720, 1454}}, sa_flags = 269542432, sa_restorer = 0x5fc8}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007f57100e6415 in ?? () from /usr/lib64/libmongoc-priv.so.0
No symbol table info available.
#3  0x00007f57100e6838 in mongoc_cluster_stream_for_server () from /usr/lib64/libmongoc-priv.so.0
No symbol table info available.
#4  0x00007f57100e11a6 in _mongoc_client_kill_cursor () from /usr/lib64/libmongoc-priv.so.0
No symbol table info available.
#5  0x00007f57100ed028 in _mongoc_cursor_destroy () from /usr/lib64/libmongoc-priv.so.0
No symbol table info available.
#6  0x00007f571033e7a2 in php_phongo_cursor_free (cursor=0x331e5c0) at /usr/src/debug/php-pecl-mongodb-1.1.7/NTS/php_phongo.c:2156
No locals.
#7  0x00007f5710335bd1 in php_phongo_cursor_free_object (object=0x331e5c0) at /usr/src/debug/php-pecl-mongodb-1.1.7/NTS/src/MongoDB/Cursor.c:226
        intern = 0x331e5c0
#8  0x00000000005deedc in zend_objects_store_free_object_storage ()
No symbol table info available.
#9  0x00000000005a7d53 in ?? ()
No symbol table info available.
#10 0x00000000005b5e32 in ?? ()
No symbol table info available.
#11 0x0000000000555cac in php_request_shutdown ()
No symbol table info available.
#12 0x000000000066ece9 in ?? ()
No symbol table info available.
#13 0x0000003b20e1ed5d in __libc_start_main (main=0x66dfb0, argc=2, ubp_av=0x7fffcaf13238, init=<value optimized out>, fini=<value optimized out>, rtld_fini=<value optimized out>, stack_end=0x7fffcaf13228)
    at libc-start.c:226
        result = <value optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 5625048110248152781, 4329616, 140736598192688, 0, 0, -5625072254190432563, 5649324518295129805}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
              0x1}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <value optimized out>
#14 0x00000000004210b9 in _start ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.

We are going to uninstall the old php-pecl-mongo extension and see if that fixes it. After that we will upgrade to 1.1.8 to see if that helps.

  • Remi
  • Remi
  • Administrateur
  • Offline
  • From: Champagne...
  • Registered: 2009-04-28
  • Posts: 3,834

Re: PHP crashes in shutdown handler with php-pecl-mongodb

> Could not find debuginfo for main pkg: mongo-c-driver-1.3.5-1.el6.remi.x86_64

=> http://rpms.remirepo.net/enterprise/6/d … ginfo.html

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Remi's Website

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

Update: We uninstalled php-pecl-mongo entirely and it is still occurring. I am going to continue trying to get the debug symbols installed so I can get a complete trace.

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

Here is the updated trace with debug symbols from mongo-c-driver

(gdb) bt full
#0  0x0000003b20e32625 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
        resultvar = 0
        pid = <value optimized out>
        selftid = <value optimized out>
#1  0x0000003b20e33e05 in abort () at abort.c:92
        save_stage = 2
        act = {__sigaction_handler = {sa_handler = 0x127121d40f60, sa_sigaction = 0x127121d40f60}, sa_mask = {__val = {253958350976, 20227416, 20215312, 0, 140736598182256, 1, 253955669547, 206158430256,
              140736598180856, 140736598180640, 0, 12821671817868660429, 5625048504752721613, 0, 140011908422720, 1454}}, sa_flags = 269542432, sa_restorer = 0x5fc8}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007f57100e6415 in mongoc_cluster_fetch_stream_pooled (cluster=0x134a558, sd=0x312a140, reconnect_ok=false, error=0x0) at src/mongoc/mongoc-cluster.c:1544
        stream = 0x7fffcaf10970
        topology = <value optimized out>
        cluster_node = <value optimized out>
        timestamp = <value optimized out>
#3  _mongoc_cluster_stream_for_server_description (cluster=0x134a558, sd=0x312a140, reconnect_ok=false, error=0x0) at src/mongoc/mongoc-cluster.c:1365
        topology = <value optimized out>
        server_stream = <value optimized out>
#4  0x00007f57100e6838 in mongoc_cluster_stream_for_server (cluster=0x134a558, server_id=3, reconnect_ok=false, error=0x0) at src/mongoc/mongoc-cluster.c:1422
        topology = <value optimized out>
        sd = 0x312a140
        server_stream = 0x0
        __func__ = "mongoc_cluster_stream_for_server"
#5  0x00007f57100e11a6 in _mongoc_client_kill_cursor (client=0x134a540, server_id=<value optimized out>, cursor_id=3908828641047568464, db=0x7fffcaf10970 "opensky_devo", collection=0x3357485 "sellables")
    at src/mongoc/mongoc-client.c:1282
        server_stream = <value optimized out>
        __func__ = "_mongoc_client_kill_cursor"
#6  0x00007f57100ed028 in _mongoc_cursor_destroy (cursor=0x33572d0) at src/mongoc/mongoc-cursor.c:273
        db = "opensky_devo\000\000\000\000h\001\000\000\000\000\000\000\202\342X\000\000\000\000\000\360\b\000\000\000\000\000\000\202\342X\000\000\000\000\000\220 \000\000\000\000\000\000\202\342X\000\000\000\000\000\360\n\000\000\000\000\000\000\202\342X\000\000\000\000\000\370\005\000\000\000\000\000\000\202\342X\000\000\000\000\000`\000\000\000\000\000\000\000\202\342X\000\000\000\000\000\060\343\216\002", '\000' <repeats 11 times>
        __func__ = "_mongoc_cursor_destroy"
#7  0x00007f571033e7a2 in php_phongo_cursor_free (cursor=0x331e5c0) at /usr/src/debug/php-pecl-mongodb-1.1.7/NTS/php_phongo.c:2156
No locals.
#8  0x00007f5710335bd1 in php_phongo_cursor_free_object (object=0x331e5c0) at /usr/src/debug/php-pecl-mongodb-1.1.7/NTS/src/MongoDB/Cursor.c:226
        intern = 0x331e5c0
#9  0x00000000005deedc in zend_objects_store_free_object_storage ()
No symbol table info available.
#10 0x00000000005a7d53 in ?? ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#11 0x00000000005b5e32 in ?? ()
No symbol table info available.
#12 0x0000000000555cac in php_request_shutdown ()
No symbol table info available.
#13 0x000000000066ece9 in ?? ()
No symbol table info available.
#14 0x0000003b20e1ed5d in __libc_start_main (main=0x66dfb0, argc=2, ubp_av=0x7fffcaf13238, init=<value optimized out>, fini=<value optimized out>, rtld_fini=<value optimized out>, stack_end=0x7fffcaf13228)
    at libc-start.c:226
        result = <value optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 5625048110248152781, 4329616, 140736598192688, 0, 0, -5625072254190432563, 5649324518295129805}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
              0x1}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <value optimized out>
#15 0x00000000004210b9 in _start ()
No symbol table info available.

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

I am currently looking in to why mongoc_cluster_fetch_stream_pooled() shows up in that trace and why the extension isn't using libmongoc's single_threaded mode.

From Jeremy Mikola, one of the authors: "ext-mongodb uses https://github.com/mongodb/mongo-c-driv … ent.c#L617 to create the libmongoc client, and that creates a single_threaded topology via github.com/mongodb/mongo-c-driver/blob/1.3.5/src/mongoc/mongoc-topology.c#L131 -- I see no way that boolean would be false for your backtrace but it appears it is."

  • Remi
  • Remi
  • Administrateur
  • Offline
  • From: Champagne...
  • Registered: 2009-04-28
  • Posts: 3,834

Re: PHP crashes in shutdown handler with php-pecl-mongodb

Have you open an upstream bug report  ? (Would be useful, as you have a nice backtrace)

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Remi's Website

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

Thanks Remi. I have created a ticket with MongoDB here https://jira.mongodb.org/browse/PHPC-727

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

We moved the ticket to mongodb commercial support since we have a support contract with them so you all won't be able to see it now. I will report back here what we find out. Thanks for all your help so far!

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

I was able to reproduce the crash with a script like the following:


<?php

use Symfony\Component\ClassLoader\ApcClassLoader;

$loader = require_once __DIR__.'/opensky/bootstrap.php.cache';
$loader = new ApcClassLoader('devo', $loader);
$loader->register(true);

$mongo = new \MongoClient('mongodb://localhost:27017');

$db = $mongo->selectDB('opensky_devo');

$collection = $db->selectCollection('sellables');

$cursor = $collection->find([], []);
$cursor->batchSize(2);
$cursor->limit(3);

$cursor->next();

I have worked around the issue for now by changing the queries so they do not use next() or hasNext() so we don't have cursors open that don't have all the results exhausted.

  • Registered: 2016-07-07
  • Posts: 11

Re: PHP crashes in shutdown handler with php-pecl-mongodb

here are the tickets for this issue https://jira.mongodb.org/browse/PHPC-732 and https://jira.mongodb.org/browse/PHPC-733