Discussion:
[asterisk-users] Call Recording and Posting
Dan Journo
2009-10-11 23:15:03 UTC
Permalink
Hello,



I'm working on a call recording solution. I would like recordings to
either be automatically uploaded via FTP, or posted to a URL for
processing by our main server.



Is Asterisk capable of doing this or will I have to create a separate
application that monitors a temp directory for new recordings?



I ask because I don't have any experience in Linux programming, so I
won't be able to create a monitoring program on my own.



Many thanks

Dan Journo
Elliot Otchet
2009-10-12 02:36:17 UTC
Permalink
Dan,

You can do this directly in the dialplan. See the System command. It allows you to call any program on the system (ftp, scp, mv, etc). Keep in mind that depending on the volume of calls you're handling, you might run into I/O issues on the disk side. If you're talking about a machine under enough load, you might need another alternative.

-Elliot

From: asterisk-users-***@lists.digium.com [mailto:asterisk-users-***@lists.digium.com] On Behalf Of Dan Journo
Sent: Sunday, October 11, 2009 7:15 PM
To: asterisk-***@lists.digium.com
Subject: [asterisk-users] Call Recording and Posting

Hello,

I'm working on a call recording solution. I would like recordings to either be automatically uploaded via FTP, or posted to a URL for processing by our main server.

Is Asterisk capable of doing this or will I have to create a separate application that monitors a temp directory for new recordings?

I ask because I don't have any experience in Linux programming, so I won't be able to create a monitoring program on my own.

Many thanks
Dan Journo


________________________________
This message is intended only for the use of the individual (s) or entity to which it is addressed and may contain information that is privileged, confidential, and/or proprietary to Calling Circles LLC and its affiliates. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, forwarding or copying of this communication is prohibited without the express permission of the sender. If you have received this communication in error, please notify the sender immediately and delete the original message.
Steve Edwards
2009-10-12 02:57:04 UTC
Permalink
Post by Dan Journo
I'm working on a call recording solution. I would like recordings to
either be automatically uploaded via FTP, or posted to a URL for
processing by our main server.
Is Asterisk capable of doing this or will I have to create a separate
application that monitors a temp directory for new recordings?
I ask because I don't have any experience in Linux programming, so I
won't be able to create a monitoring program on my own.
There is no built in facility -- but there are all the "parts."

There is the curl() application, but I don't know if it exposes enough
"curl" to upload files.

There is the system() application which will let you execute any command
line you can construct.

There is the agi() application which lets an external program interact
with the dialplan.

Monitoring a temp directory with an external program would be the "worst"
way.

Personally, I would wrap up the entire "call recording solution" in an AGI
so you have a full featured language (my preference is C) and can hide all
the ugly details and keep your dialplan simple and maintainable.

I've done these kind of applications where either a "control file" needed
to be written and uploaded with the recording or a database needed
updating. Both of these can get ugly in a dialplan.
--
Thanks in advance,
-------------------------------------------------------------------------
Steve Edwards ***@sedwards.com Voice: +1-760-468-3867 PST
Newline Fax: +1-760-731-3000
Dovid Bender
2009-10-12 08:27:59 UTC
Permalink
You can try using NFS. Also you can pay some one to write script that would move the files over on hang up.
----- Original Message -----
From: Dan Journo
To: asterisk-***@lists.digium.com
Sent: Monday, October 12, 2009 01:15
Subject: [asterisk-users] Call Recording and Posting


Hello,



I'm working on a call recording solution. I would like recordings to either be automatically uploaded via FTP, or posted to a URL for processing by our main server.



Is Asterisk capable of doing this or will I have to create a separate application that monitors a temp directory for new recordings?



I ask because I don't have any experience in Linux programming, so I won't be able to create a monitoring program on my own.



Many thanks

Dan Journo





------------------------------------------------------------------------------


_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

AstriCon 2009 - October 13 - 15 Phoenix, Arizona
Register Now: http://www.astricon.net

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Ivan Stepaniuk
2009-10-12 11:44:27 UTC
Permalink
Post by Dan Journo
I'm working on a call recording solution. I would like recordings to
either be automatically uploaded via FTP, or posted to a URL for
processing by our main server.
Is Asterisk capable of doing this or will I have to create a separate
application that monitors a temp directory for new recordings?
As said by others, there is no such built-in capability
Post by Dan Journo
I ask because I don't have any experience in Linux programming, so I
won't be able to create a monitoring program on my own.
It is really important for you that the recordings are available asap on
your FTP destination? I had a similar task and I found problematic to
upload the files from inside the dialplan, Is not that it can't be done,
but if the FTP is slow and your number of connections limited, you may
run into a problem with simultaneous calls ending and asterisk trying to
upload 20 files at the same time.

In my case the files could be uploaded every hour, so I made a simple
bash script that uploads the new files to the FTP using 'curlftpfs', a
nice command that mounts the remote FTP on a local mount point using
FUSE, then the script just moves the files from the local folder to the
FTP, and voila. Asterisk just takes care of moving recordings that ended
to the desired path. I can post the bash script if you are interested.
--
Iván Stepaniuk
Alba Fotónica S. L.
www.albafotonica.com
Dan Journo
2009-10-12 12:02:44 UTC
Permalink
Hi Iván,

Thank you for replying. I hadn't thought about the problem of simultaneous calls. It would be a problem if a number of calls ended at the same time.

If you can post it, the script would really be helpful as I'm only a beginner with Linux.

Many thanks
Dan Journo


-----Original Message-----
From: asterisk-users-***@lists.digium.com [mailto:asterisk-users-***@lists.digium.com] On Behalf Of Ivan Stepaniuk
Sent: 12 October 2009 12:44
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [asterisk-users] Call Recording and Posting
Post by Dan Journo
I'm working on a call recording solution. I would like recordings to
either be automatically uploaded via FTP, or posted to a URL for
processing by our main server.
Is Asterisk capable of doing this or will I have to create a separate
application that monitors a temp directory for new recordings?
As said by others, there is no such built-in capability
Post by Dan Journo
I ask because I don't have any experience in Linux programming, so I
won't be able to create a monitoring program on my own.
It is really important for you that the recordings are available asap on
your FTP destination? I had a similar task and I found problematic to
upload the files from inside the dialplan, Is not that it can't be done,
but if the FTP is slow and your number of connections limited, you may
run into a problem with simultaneous calls ending and asterisk trying to
upload 20 files at the same time.

In my case the files could be uploaded every hour, so I made a simple
bash script that uploads the new files to the FTP using 'curlftpfs', a
nice command that mounts the remote FTP on a local mount point using
FUSE, then the script just moves the files from the local folder to the
FTP, and voila. Asterisk just takes care of moving recordings that ended
to the desired path. I can post the bash script if you are interested.
--
Iván Stepaniuk
Alba Fotónica S. L.
www.albafotonica.com


_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

AstriCon 2009 - October 13 - 15 Phoenix, Arizona
Register Now: http://www.astricon.net

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Ivan Stepaniuk
2009-10-12 21:12:02 UTC
Permalink
Post by Dan Journo
Thank you for replying. I hadn't thought about the problem of simultaneous calls. It would be a problem if a number of calls ended at the same time.
If you can post it, the script would really be helpful as I'm only a beginner with Linux
The script is very simple and far from complete, it just moves the
content into the mounted FTP directory. It has some verbose output as it
is run from inside another script that redirects the output to a log file.
The script has a password inside so remember to 'chown root' and
'chmod 700' the file to protect it from other users.
You have to set up a cron so the script is run every hour, normally
putting the script or a link to it inside '/etc/cron.hourly' is enough.
There is also a /etc/crontab file you can use to setup something more
complicated if needed (ie: runing it every 2 hours, running at tea
time...). read 'man cron', and 'man crontab'.
You also need to install the magic part, 'curlftps'. in Debian
that's the name of the package too. I use version curlftpfs 0.9.1
libcurl/7.18.2 fuse/2.5
Be careful that in *nix, file.WAV and file.wav are different files.

Here is the script:

#!/bin/sh

MOUNT_POINT="/mnt/remote_ftp"
FTP_HOST="www.ftphost.com/htdocs/recordings"
FTP_USER="ftpusername:difficultpassword"
RECORDINGS="/var/spool/asterisk/monitor/upload"

echo "Starting upload `date` "
echo "Connecting to $FTP_HOST..."
curlftpfs -o user="$FTP_USER" "$FTP_HOST" "$MOUNT_POINT"

echo "Transfering `du -hc "$RECORDINGS"/*.wav | grep total` ..."
mv -vf "$RECORDINGS"/*.wav "$MOUNT_POINT"

echo "Disconnecting."
umount "$MOUNT_POINT"

exit 0
--
Iván Stepaniuk
Alba Fotónica S. L.
www.albafotonica.com
Dan Journo
2009-10-12 21:38:38 UTC
Permalink
Thanks for that.

I really appreciate it!

Dan


-----Original Message-----
From: asterisk-users-***@lists.digium.com [mailto:asterisk-users-***@lists.digium.com] On Behalf Of Ivan Stepaniuk
Sent: 12 October 2009 22:12
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [asterisk-users] Call Recording and Posting
Post by Dan Journo
Thank you for replying. I hadn't thought about the problem of simultaneous calls. It would be a problem if a number of calls ended at the same time.
If you can post it, the script would really be helpful as I'm only a beginner with Linux
The script is very simple and far from complete, it just moves the
content into the mounted FTP directory. It has some verbose output as it
is run from inside another script that redirects the output to a log file.
The script has a password inside so remember to 'chown root' and
'chmod 700' the file to protect it from other users.
You have to set up a cron so the script is run every hour, normally
putting the script or a link to it inside '/etc/cron.hourly' is enough.
There is also a /etc/crontab file you can use to setup something more
complicated if needed (ie: runing it every 2 hours, running at tea
time...). read 'man cron', and 'man crontab'.
You also need to install the magic part, 'curlftps'. in Debian
that's the name of the package too. I use version curlftpfs 0.9.1
libcurl/7.18.2 fuse/2.5
Be careful that in *nix, file.WAV and file.wav are different files.

Here is the script:

#!/bin/sh

MOUNT_POINT="/mnt/remote_ftp"
FTP_HOST="www.ftphost.com/htdocs/recordings"
FTP_USER="ftpusername:difficultpassword"
RECORDINGS="/var/spool/asterisk/monitor/upload"

echo "Starting upload `date` "
echo "Connecting to $FTP_HOST..."
curlftpfs -o user="$FTP_USER" "$FTP_HOST" "$MOUNT_POINT"

echo "Transfering `du -hc "$RECORDINGS"/*.wav | grep total` ..."
mv -vf "$RECORDINGS"/*.wav "$MOUNT_POINT"

echo "Disconnecting."
umount "$MOUNT_POINT"

exit 0
--
Iván Stepaniuk
Alba Fotónica S. L.
www.albafotonica.com


_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

AstriCon 2009 - October 13 - 15 Phoenix, Arizona
Register Now: http://www.astricon.net

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Steve Edwards
2009-10-12 22:26:38 UTC
Permalink
Post by Dan Journo
On Behalf Of Ivan Stepaniuk
The script is very simple and far from complete, it just moves the
content into the mounted FTP directory. It has some verbose output as it
is run from inside another script that redirects the output to a log file.
What happens if the script is run while a recording is being written? Will
it copy the incomplete file and then delete it?
Post by Dan Journo
The script has a password inside so remember to 'chown root' and
'chmod 700' the file to protect it from other users.
If you used sshfs you could use public keys and eliminate the password
hassles.

Personally, I'd still vote for uploading the file at the completion of the
recording via an AGI.
--
Thanks in advance,
-------------------------------------------------------------------------
Steve Edwards ***@sedwards.com Voice: +1-760-468-3867 PST
Newline Fax: +1-760-731-3000
Dan Journo
2009-10-12 23:08:23 UTC
Permalink
Hi,

To avoid the problem of deleting/copying calls that are still being
recorded, I could record the call into a temp directory.
Then using the dial plan, I could copy the temp recording into the ftp
root directory once the call has ended.

Dan




Thank you for contacting Kesher Communications Ltd.
IT Maintenance Clients can now receive a faster response by using our
Live Chat and Support Service: Click Here

This email and any files transmitted with it are confidential and
intended solely for the recipient(s). If you are not the named addressee
you should not disseminate, copy or alter this email. Under no
circumstances may this email be distributed without written permission
from the sender. Warning: Although the Company has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use
of this email or attachments. All prices exclude VAT unless otherwise
stated.


-----Original Message-----
From: asterisk-users-***@lists.digium.com
[mailto:asterisk-users-***@lists.digium.com] On Behalf Of Steve
Edwards
Sent: 12 October 2009 23:27
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [asterisk-users] Call Recording and Posting
Post by Dan Journo
On Behalf Of Ivan Stepaniuk
The script is very simple and far from complete, it just moves the
content into the mounted FTP directory. It has some verbose output as it
is run from inside another script that redirects the output to a log file.
What happens if the script is run while a recording is being written?
Will
it copy the incomplete file and then delete it?
Post by Dan Journo
The script has a password inside so remember to 'chown root' and
'chmod 700' the file to protect it from other users.
If you used sshfs you could use public keys and eliminate the password
hassles.

Personally, I'd still vote for uploading the file at the completion of
the
recording via an AGI.
--
Thanks in advance,
------------------------------------------------------------------------
-
Steve Edwards ***@sedwards.com Voice: +1-760-468-3867
PST
Newline Fax:
+1-760-731-3000

_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

AstriCon 2009 - October 13 - 15 Phoenix, Arizona
Register Now: http://www.astricon.net

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Steve Edwards
2009-10-13 01:14:36 UTC
Permalink
Post by Dan Journo
To avoid the problem of deleting/copying calls that are still being
recorded, I could record the call into a temp directory. Then using the
dial plan, I could copy the temp recording into the ftp root directory
once the call has ended.
True, but if you need to execute a process at the end of the call, why not
make it an AGI and hide all the ugly details and keep your dialplan nice
and clean and shiny and maintainable?

Your recordings will be "instantly" available and the correct operation of
your system does not depend on an externally scheduled external process
involving clear-text passwords and obscure packages. Your successor will
thank you :)
--
Thanks in advance,
-------------------------------------------------------------------------
Steve Edwards ***@sedwards.com Voice: +1-760-468-3867 PST
Newline Fax: +1-760-731-3000
Ivan Stepaniuk
2009-10-13 07:00:47 UTC
Permalink
Post by Steve Edwards
Post by Dan Journo
To avoid the problem of deleting/copying calls that are still being
recorded, I could record the call into a temp directory. Then using the
dial plan, I could copy the temp recording into the ftp root directory
once the call has ended.
True, but if you need to execute a process at the end of the call, why not
make it an AGI and hide all the ugly details and keep your dialplan nice
and clean and shiny and maintainable?
Your recordings will be "instantly" available and the correct operation of
your system does not depend on an externally scheduled external process
involving clear-text passwords and obscure packages. Your successor will
thank you :)
This is true, doing everything from inside an AGI script would be nicer,
the ugly part comes if you are tied to an old and ugly FTP server,
specially if its from a hosting provider that limits the connection
count to 2, or so. AGI+sshfs/scp/nfs/whateverfs... would be much more
clean&shiny (tm).
--
Iván Stepaniuk
Alba Fotónica S. L.
www.albafotonica.com
Elliot Otchet
2009-10-13 15:51:25 UTC
Permalink
Steve's spot on about wanting to move the calls when they're available. We're the instant gratification society.

Anytime I've linked multiple independent systems together I've always had to plan for one system being offline - either for maintenance, upgrades, etc. so please consider that in your solution. Your successor will thank you for that too! If you were only posting per call via AGI, then you might have some issues if the receiving server was unavailable. Ivan's solution handles that cleanly (it would simply try moving the file again when the script is invoked again). What to do?...Combine approaches!!

You could add some error checking to Ivan's script that tests if the file system is mounted (or another copy of the script is running) before invoking the other commands and exit if it is (e.g. look at the exit status code of curlftpfs, maybe?).

Doing so gives you three major benefits:
Prevents multiple copies of the script from running and trying to process the same file (low probability, but theoretical)
Lets you call the script more frequently without having to worry about multiple processes running simultaneously (the need to manage your concurrency).
Gives you a way to safely call this on a per call basis from the dialplan (right after the h,1,System(move the file to the upload directory)) to get the trigger for instant gratification.

To Steve's other point, you could put all of this into an AGI program/script, but you'll still also need a fallback mechanism to actually copy the files to the remote server in the event that it is unavailable/unreachable. To me, having two lines in the dialplan versus one is no big deal. Just make sure you add comments for it so your successors know the logic behind the code.

Just some more thoughts.

-Elliot

-----Original Message-----
From: asterisk-users-***@lists.digium.com [mailto:asterisk-users-***@lists.digium.com] On Behalf Of Ivan Stepaniuk
Sent: Tuesday, October 13, 2009 3:01 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [asterisk-users] Call Recording and Posting
Post by Steve Edwards
Post by Dan Journo
To avoid the problem of deleting/copying calls that are still being
recorded, I could record the call into a temp directory. Then using the
dial plan, I could copy the temp recording into the ftp root directory
once the call has ended.
True, but if you need to execute a process at the end of the call, why not
make it an AGI and hide all the ugly details and keep your dialplan nice
and clean and shiny and maintainable?
Your recordings will be "instantly" available and the correct operation of
your system does not depend on an externally scheduled external process
involving clear-text passwords and obscure packages. Your successor will
thank you :)
This is true, doing everything from inside an AGI script would be nicer,
the ugly part comes if you are tied to an old and ugly FTP server,
specially if its from a hosting provider that limits the connection
count to 2, or so. AGI+sshfs/scp/nfs/whateverfs... would be much more
clean&shiny (tm).

--
Iván Stepaniuk
Alba Fotónica S. L.
www.albafotonica.com


_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

AstriCon 2009 - October 13 - 15 Phoenix, Arizona
Register Now: http://www.astricon.net

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users

This message is intended only for the use of the individual (s) or entity to which it is addressed and may contain information that is privileged, confidential, and/or proprietary to Calling Circles LLC and its affiliates. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, forwarding or copying of this communication is prohibited without the express permission of the sender. If you have received this communication in error, please notify the sender immediately and delete the original message.
Steve Edwards
2009-10-13 16:48:48 UTC
Permalink
Post by Elliot Otchet
To Steve's other point, you could put all of this into an AGI
program/script, but you'll still also need a fallback mechanism to
actually copy the files to the remote server in the event that it is
unavailable/unreachable. To me, having two lines in the dialplan versus
one is no big deal. Just make sure you add comments for it so your
successors know the logic behind the code.
My AGI logs to syslog so I can track and follow up on failures.

The simple task of "can you upload a recording" quickly spiraled out of
dialplan territory.

It kind of went like this:

1) Can you upload the recording?

2) It takes to long to upload and download the recording. Can you encode
it to a low bitrate WMA?

3) Some of the recordings are too [loud|quiet]. Can you normalize them?

4) There's too much dead air at the [beginning|end] of the recording. Can
you trim off the cruft?

5) It takes too long to listen to all of the questions. (My recordings are
a sequence of pre-recorded questions and caller voice and DTMF responses).
Can you also upload a recording of just the responses?

6) Can you save all the DTMF responses in a database so we can do some
reporting.
--
Thanks in advance,
-------------------------------------------------------------------------
Steve Edwards ***@sedwards.com Voice: +1-760-468-3867 PST
Newline Fax: +1-760-731-3000
Ivan Stepaniuk
2009-10-13 06:53:10 UTC
Permalink
Post by Dan Journo
Thanks for that.
I really appreciate it!
Dan
As pointed by the follow-ups, note that the recordings are not taken
from the monitor but from an upload folder inside, the dialplan takes
care to move there the files for the ended files only issuing a 'System'
command.
--
Iván Stepaniuk
Alba Fotónica S. L.
www.albafotonica.com
Loading...