Problems updating from 17.13 to 17.16 and missing files

Added by C G 11 months ago

The last time I was able to successfully update was 17.13. I skipped 17.14 but tried 17.15 and now 17.16. After applying the update and having it reboot through Vultr this is what I get in the logs:

starting xivo-ctid-ng ...Job for xivo-ctid-ng.service failed. See 'systemctl status xivo-ctid-ng.service' and 'journalctl -xn' for details.

2017-11-28 23:01:01,528 [2653] (WARNING) (xivo_ctid_ng.controller): Stopping xivo-ctid-ng: SIGTERM
2017-11-28 23:01:01,536 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:01] ENGINE Bus STOPPING
2017-11-28 23:01:01,551 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:01] ENGINE HTTP Server cheroot.wsgi.Server(('', 9500)) shut down
2017-11-28 23:01:01,559 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:01] ENGINE HTTP Server cheroot.wsgi.Server(('', 9501)) shut down
2017-11-28 23:01:01,559 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:01] ENGINE Stopped thread '_TimeoutMonitor'.
2017-11-28 23:01:01,559 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:01] ENGINE Bus STOPPED
2017-11-28 23:01:01,559 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:01] ENGINE Bus EXITING
2017-11-28 23:01:01,560 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:01] ENGINE Bus EXITED
2017-11-28 23:01:01,593 [2653] (INFO) (service_discovery): Deregistering xivo-ctid-ng from Consul services: 73f6580a-3a92-4164-ab27-926c6019ebcf
2017-11-28 23:01:01,719 [2653] (INFO) (xivo_ctid_ng.controller): xivo-ctid-ng stopping...
2017-11-28 23:01:01,799 [2653] (ERROR) (xivo_ctid_ng.core.ari_): Error while listening for ARI events: [Errno 32] Broken pipe
2017-11-28 23:01:02,692 [2653] (INFO) (cherrypy.error): [28/Nov/2017:23:01:02] ENGINE Waiting for child threads to terminate...

So at this point I restore my previous snapshot to get things up and running again. xivo-ctid-ng.log after booting:

File "/usr/lib/python2.7/dist-packages/xivo_confd_client/", line 35, in check_response
File "/usr/lib/python2.7/dist-packages/requests/", line 825, in raise_for_status
raise HTTPError(http_error_msg, response=self)
HTTPError: 401 Client Error: UNAUTHORIZED
2017-11-29 00:05:44,258 [2670] (INFO) (cherrypy.error): [29/Nov/2017:00:05:44] ENGINE Bus STARTING
2017-11-29 00:05:44,270 [2670] (INFO) (cherrypy.error): [29/Nov/2017:00:05:44] ENGINE Started monitor thread '_TimeoutMonitor'.
2017-11-29 00:05:44,478 [2670] (INFO) (cherrypy.error): [29/Nov/2017:00:05:44] ENGINE Serving on unknown interface (dynamic?)
2017-11-29 00:05:44,680 [2670] (INFO) (cherrypy.error): [29/Nov/2017:00:05:44] ENGINE Serving on unknown interface (dynamic?)
2017-11-29 00:05:44,680 [2670] (INFO) (cherrypy.error): [29/Nov/2017:00:05:44] ENGINE Bus STARTED
2017-11-29 00:05:46,315 [2670] (ERROR) (xivo.rest_api_helpers): Unauthorized: {'invalid_token': ''}
2017-11-29 00:05:46,321 [2670] (INFO) (xivo_ctid_ng): ( GET https://localhost:9500/1.0/status 401
2017-11-29 00:05:46,324 [2670] (INFO) (service_discovery): Registering xivo-ctid-ng on Consul as 5ab156c3-f2d6-44b5-e8fa-81535b4864c8 with

I obfuscated the consul ID and IP address but wanted to show that they are different so I'm not sure if that's supposed to be the case. In addition to that, I noticed at some point pbxstatus stopped working. The file being called is missing as /usr/local/sbin only contains 3 files for me: iptables-custom, iptables-knock and iptables-restart. Clearly there are other issues I need to address. I do recall seeing some ACL issues. Would doing a xivo-update-keys cause any issues if it appears everything in 17.13 is working now?

I also tried to run vimdiff and can't seem to find that file either so if someone has an exact location I can verify if it is truly missing.

Replies (4)

RE: Problems updating from 17.13 to 17.16 and missing files - Added by James Cardenas 10 months ago

How exactly are you upgrading wazo?

RE: Problems updating from 17.13 to 17.16 and missing files - Added by C G 10 months ago


Initially I believe I started with 17.03 and couldn't upgrade until I did
xivo-dist phoenix

I've just been using wazo-upgrade since then. But because of the issues I've been having I even tried


which is just a slight modification wazo-upgrade

RE: Problems updating from 17.13 to 17.16 and missing files - Added by Sébastien Le Moal 10 months ago

Hello, a similar problem justed resolve in the forum french section.
Verify you codec G729 it's not activate for can start ctid service.

Sébastien Le Moal
Fabricant et Fournisseur d'appliance Routeur, VOIP
Distributeur et importateur Fanvil France

RE: Problems updating from 17.13 to 17.16 and missing files - Added by C G 10 months ago

Thanks for the response, Sebastien. Unfortunately it doesn't appear I'm using G729 codec. I've checked SIP Protocol in General Settings and Trunk management. Is there another place I should check? I rolled back to 17.13 after the failed 17.16 update. So unless it spontaneously appears between my updates, I haven't checked 17.16 to see if the offending codec is listed.

In the meantime, I found the portion of the script that installs pbxstatus and restored it. But when I run it, it just sits there and doesn't give me the old familiar ANSI colored screen so i have to do a CTRL-C to exit. So I just renamed the file.