Discussion:
[asterisk-users] Simplest way of executing a non-blocking (async) python AGI script?
Jonathan H
2017-06-30 17:11:08 UTC
Permalink
I use a python AGI which pulls some info from a web service, which should
take half a second.

Sometimes, it takes 5-10 seconds which blocks the dialplan execution, but
the dialplan should continue immediately as it's not dependent on the
AGI/web service data.

What's the simplest, easiest quickest least-code way of firing off an AGI
with some variable, and then returning to the dialplan?

I've seen people talking about fastAGI, stasis, python ASYNC... all seems
rather complicated. I feel I must be missing something embarrassingly
obvious - isn't there just the equivalent of the unix shell's "&"?!

Thanks.

(I'm using pyst2 if it matters)
Antony Stone
2017-06-30 17:23:04 UTC
Permalink
Post by Jonathan H
I use a python AGI which pulls some info from a web service, which should
take half a second.
Sometimes, it takes 5-10 seconds which blocks the dialplan execution, but
the dialplan should continue immediately as it's not dependent on the
AGI/web service data.
What's the simplest, easiest quickest least-code way of firing off an AGI
with some variable, and then returning to the dialplan?
Write your python code to fork() the lookup to a child process, and let the
parent return immediately to Asterisk.
Post by Jonathan H
I've seen people talking about fastAGI, stasis, python ASYNC... all seems
rather complicated. I feel I must be missing something embarrassingly
obvious - isn't there just the equivalent of the unix shell's "&"?!
Not inside Asterisk, no.


Antony.
--
The words "e pluribus unum" on the Great Seal of the United States are from a
poem by Virgil entitled "Moretum", which is about cheese and garlic salad
dressing.

Please reply to the list;
please *don't* CC me.
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
J Montoya or A J Stiles
2017-06-30 18:59:08 UTC
Permalink
Post by Jonathan H
What's the simplest, easiest quickest least-code way of firing off an AGI
with some variable, and then returning to the dialplan?
You have to use the "fork" command. This starts a copy of the process with
all the same internal state including variables and filehandles. The command
returns a non-zero value (which is the PID of the child process; you may need
this, if you plan to outlive your children and have to clear their entries
from the process table) to the parent process, and zero to the child process.
So in the parent, you exit and return to the dialplan; and in the child, you
close STDIN, STDOUT and STDERR (so no process is waiting for you to produce
output), then just take your time doing what you have to. The parent is
already long dead by this time, so exiting goes nowhere.
Post by Jonathan H
I've seen people talking about fastAGI, stasis, python ASYNC... all seems
rather complicated. I feel I must be missing something embarrassingly
obvious - isn't there just the equivalent of the unix shell's "&"?!
Yes, fork! That is what the "&" operator is using "under the bonnet".
--
JKLM

Note: Originating address only accepts e-mail from list! If replying off-
list, change address to asterisk1list at earthshod dot co dot uk .
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Jonathan H
2017-06-30 21:23:44 UTC
Permalink
OK, I give up and come grovelling, "Fork" was suggested at 18:23, it's
now 22:20 and I have been through 4 different methods, all block with
a 2 second delay before returning to dialplan.

Here are just some of the examples I have tried, as as per the
suggestions, I am closing all possible outputs in the forked process.

https://docs.python.org/3.5/library/multiprocessing.html
https://docs.python.org/3.5/library/multiprocessing.html?highlight=multiprocessing#multiprocessing.Process.join
https://stackoverflow.com/questions/19747371/python-exit-commands-why-so-many-and-when-should-each-be-used
https://stackoverflow.com/questions/27624850/launch-a-completely-independent-process
https://stackoverflow.com/questions/13612434/why-are-the-methods-sys-exit-exit-raise-systemexit-not-working
https://stackoverflow.com/questions/43280947/os-fork-share-local-variable-with-parent
https://stackoverflow.com/questions/24052217/may-someone-explain-the-following-os-fork-example-to-me
http://www.python-course.eu/forking.php
https://pymotw.com/3/subprocess/
http://code.activestate.com/recipes/186101-really-closing-stdin-stdout-stderr/

This is the most likely looking code based on the examples. I would
really, really appreciate a couple of pointers as to where I might be
going wrong:

#! /usr/bin/env python3
# -*- coding: utf-8 -*-

import multiprocessing as mp
import time
import sys
import os

#from asterisk.agi import AGI
#agi = AGI()

def f(name):
sys.stdin.close()
sys.stdout.close()
sys.stderr.close()
os.close(0) # close C's stdin stream
os.close(1) # close C's stdout stream
os.close(2) # close C's stderr stream
time.sleep(2)
f = open('/var/lib/asterisk/agi-bin/tns/testing/testout.txt', 'w')
f.write(name)
f.close()


if __name__ == '__main__':
print('before process')
mp.set_start_method('fork')
q = mp.Queue()
p = mp.Process(target=f, args=('asterisk',))
p.start()
sys.exit()

On 30 June 2017 at 19:59, J Montoya or A J Stiles
Post by J Montoya or A J Stiles
Post by Jonathan H
What's the simplest, easiest quickest least-code way of firing off an AGI
with some variable, and then returning to the dialplan?
You have to use the "fork" command. This starts a copy of the process with
all the same internal state including variables and filehandles. The command
returns a non-zero value (which is the PID of the child process; you may need
this, if you plan to outlive your children and have to clear their entries
from the process table) to the parent process, and zero to the child process.
So in the parent, you exit and return to the dialplan; and in the child, you
close STDIN, STDOUT and STDERR (so no process is waiting for you to produce
output), then just take your time doing what you have to. The parent is
already long dead by this time, so exiting goes nowhere.
Post by Jonathan H
I've seen people talking about fastAGI, stasis, python ASYNC... all seems
rather complicated. I feel I must be missing something embarrassingly
obvious - isn't there just the equivalent of the unix shell's "&"?!
Yes, fork! That is what the "&" operator is using "under the bonnet".
--
JKLM
Note: Originating address only accepts e-mail from list! If replying off-
list, change address to asterisk1list at earthshod dot co dot uk .
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Check out the new Asterisk community forum at: https://community.asterisk.org/
https://wiki.asterisk.org/wiki/display/AST/Getting+Started
asterisk-users mailing list
http://lists.digium.com/mailman/listinfo/asterisk-users
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Marcelo Terres
2017-06-30 22:18:30 UTC
Permalink
Take a look on that:

https://issues.asterisk.org/jira/browse/ASTERISK-20532

Regards,
Marcelo H. Terres <***@gmail.com>
IM: ***@jabber.mundoopensource.com.br
https://www.mundoopensource.com.br
https://twitter.com/mhterres
https://linkedin.com/in/marceloterres
Post by Jonathan H
OK, I give up and come grovelling, "Fork" was suggested at 18:23, it's
now 22:20 and I have been through 4 different methods, all block with
a 2 second delay before returning to dialplan.
Here are just some of the examples I have tried, as as per the
suggestions, I am closing all possible outputs in the forked process.
https://docs.python.org/3.5/library/multiprocessing.html
https://docs.python.org/3.5/library/multiprocessing.html?highlight=multiprocessing#multiprocessing.Process.join
https://stackoverflow.com/questions/19747371/python-exit-commands-why-so-many-and-when-should-each-be-used
https://stackoverflow.com/questions/27624850/launch-a-completely-independent-process
https://stackoverflow.com/questions/13612434/why-are-the-methods-sys-exit-exit-raise-systemexit-not-working
https://stackoverflow.com/questions/43280947/os-fork-share-local-variable-with-parent
https://stackoverflow.com/questions/24052217/may-someone-explain-the-following-os-fork-example-to-me
http://www.python-course.eu/forking.php
https://pymotw.com/3/subprocess/
http://code.activestate.com/recipes/186101-really-closing-stdin-stdout-stderr/
This is the most likely looking code based on the examples. I would
really, really appreciate a couple of pointers as to where I might be
#! /usr/bin/env python3
# -*- coding: utf-8 -*-
import multiprocessing as mp
import time
import sys
import os
#from asterisk.agi import AGI
#agi = AGI()
sys.stdin.close()
sys.stdout.close()
sys.stderr.close()
os.close(0) # close C's stdin stream
os.close(1) # close C's stdout stream
os.close(2) # close C's stderr stream
time.sleep(2)
f = open('/var/lib/asterisk/agi-bin/tns/testing/testout.txt', 'w')
f.write(name)
f.close()
print('before process')
mp.set_start_method('fork')
q = mp.Queue()
p = mp.Process(target=f, args=('asterisk',))
p.start()
sys.exit()
On 30 June 2017 at 19:59, J Montoya or A J Stiles
Post by J Montoya or A J Stiles
Post by Jonathan H
What's the simplest, easiest quickest least-code way of firing off an AGI
with some variable, and then returning to the dialplan?
You have to use the "fork" command. This starts a copy of the process with
all the same internal state including variables and filehandles. The command
returns a non-zero value (which is the PID of the child process; you may need
this, if you plan to outlive your children and have to clear their entries
from the process table) to the parent process, and zero to the child process.
So in the parent, you exit and return to the dialplan; and in the child, you
close STDIN, STDOUT and STDERR (so no process is waiting for you to produce
output), then just take your time doing what you have to. The parent is
already long dead by this time, so exiting goes nowhere.
Post by Jonathan H
I've seen people talking about fastAGI, stasis, python ASYNC... all seems
rather complicated. I feel I must be missing something embarrassingly
obvious - isn't there just the equivalent of the unix shell's "&"?!
Yes, fork! That is what the "&" operator is using "under the bonnet".
--
JKLM
Note: Originating address only accepts e-mail from list! If replying off-
list, change address to asterisk1list at earthshod dot co dot uk .
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Check out the new Asterisk community forum at: https://community.asterisk.org/
https://wiki.asterisk.org/wiki/display/AST/Getting+Started
asterisk-users mailing list
http://lists.digium.com/mailman/listinfo/asterisk-users
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Check out the new Asterisk community forum at: https://community.asterisk.org/
https://wiki.asterisk.org/wiki/display/AST/Getting+Started
asterisk-users mailing list
http://lists.digium.com/mailman/listinfo/asterisk-users
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Jonathan H
2017-07-01 07:12:09 UTC
Permalink
Oh wow - thank you SO much Marcelo.

This works perfectly - exactly as I wanted. Don't think I'd have
figured it out, though.

Well, that ticket says "not a bug" if this isn't a bug, then
"behaviour which differs dramatically from that which is expected"
would definitely be the tag.

Again, this fix works, and thank you!
Post by Marcelo Terres
https://issues.asterisk.org/jira/browse/ASTERISK-20532
Regards,
https://www.mundoopensource.com.br
https://twitter.com/mhterres
https://linkedin.com/in/marceloterres
Post by Jonathan H
OK, I give up and come grovelling, "Fork" was suggested at 18:23, it's
now 22:20 and I have been through 4 different methods, all block with
a 2 second delay before returning to dialplan.
Here are just some of the examples I have tried, as as per the
suggestions, I am closing all possible outputs in the forked process.
https://docs.python.org/3.5/library/multiprocessing.html
https://docs.python.org/3.5/library/multiprocessing.html?highlight=multiprocessing#multiprocessing.Process.join
https://stackoverflow.com/questions/19747371/python-exit-commands-why-so-many-and-when-should-each-be-used
https://stackoverflow.com/questions/27624850/launch-a-completely-independent-process
https://stackoverflow.com/questions/13612434/why-are-the-methods-sys-exit-exit-raise-systemexit-not-working
https://stackoverflow.com/questions/43280947/os-fork-share-local-variable-with-parent
https://stackoverflow.com/questions/24052217/may-someone-explain-the-following-os-fork-example-to-me
http://www.python-course.eu/forking.php
https://pymotw.com/3/subprocess/
http://code.activestate.com/recipes/186101-really-closing-stdin-stdout-stderr/
This is the most likely looking code based on the examples. I would
really, really appreciate a couple of pointers as to where I might be
#! /usr/bin/env python3
# -*- coding: utf-8 -*-
import multiprocessing as mp
import time
import sys
import os
#from asterisk.agi import AGI
#agi = AGI()
sys.stdin.close()
sys.stdout.close()
sys.stderr.close()
os.close(0) # close C's stdin stream
os.close(1) # close C's stdout stream
os.close(2) # close C's stderr stream
time.sleep(2)
f = open('/var/lib/asterisk/agi-bin/tns/testing/testout.txt', 'w')
f.write(name)
f.close()
print('before process')
mp.set_start_method('fork')
q = mp.Queue()
p = mp.Process(target=f, args=('asterisk',))
p.start()
sys.exit()
On 30 June 2017 at 19:59, J Montoya or A J Stiles
Post by J Montoya or A J Stiles
Post by Jonathan H
What's the simplest, easiest quickest least-code way of firing off an AGI
with some variable, and then returning to the dialplan?
You have to use the "fork" command. This starts a copy of the process with
all the same internal state including variables and filehandles. The command
returns a non-zero value (which is the PID of the child process; you may need
this, if you plan to outlive your children and have to clear their entries
from the process table) to the parent process, and zero to the child process.
So in the parent, you exit and return to the dialplan; and in the child, you
close STDIN, STDOUT and STDERR (so no process is waiting for you to produce
output), then just take your time doing what you have to. The parent is
already long dead by this time, so exiting goes nowhere.
Post by Jonathan H
I've seen people talking about fastAGI, stasis, python ASYNC... all seems
rather complicated. I feel I must be missing something embarrassingly
obvious - isn't there just the equivalent of the unix shell's "&"?!
Yes, fork! That is what the "&" operator is using "under the bonnet".
--
JKLM
Note: Originating address only accepts e-mail from list! If replying off-
list, change address to asterisk1list at earthshod dot co dot uk .
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Check out the new Asterisk community forum at: https://community.asterisk.org/
https://wiki.asterisk.org/wiki/display/AST/Getting+Started
asterisk-users mailing list
http://lists.digium.com/mailman/listinfo/asterisk-users
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Check out the new Asterisk community forum at: https://community.asterisk.org/
https://wiki.asterisk.org/wiki/display/AST/Getting+Started
asterisk-users mailing list
http://lists.digium.com/mailman/listinfo/asterisk-users
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Check out the new Asterisk community forum at: https://community.asterisk.org/
https://wiki.asterisk.org/wiki/display/AST/Getting+Started
asterisk-users mailing list
http://lists.digium.com/mailman/listinfo/asterisk-users
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Loading...