Recently in Server Services Category
Dear users of NABPS (be that content providers, i.e. blog owners, or content consumers, i.e. feed readers),
Please, kindly be notified that this feed aggregator is only able to process RSS2.0 and ATOM feeds that are, XML-wise, well formed. If your feed is not well formed, it will not pass into the CFF cache and, subsequently, neither into the SFS cache, to the output drivers and in the final output (be that HTML, RSS or ATOM). The net effect of this is that your content will not show up in the aggregator, even though NABPS will fetch it as dictated by If-Modified-Since and Last-Modified headers.
Please note that the two most common causes encountered for an otherwise working feed going not well formed are:
Anticipating the FAQ: no I shall not attempt to relax NABPS' XML parser in any way that would make it susceptible of accepting a not well-formed feed as valid. Standards must be obeyed, to the letter.
One, mandatory, example: Alexandru Burlacu's feed ends with the following (closing rss tag shown for clarity):
It seems the above problem has been fixed shortly after my post -- thank you for your timely reaction. This is not the first time things get fixed after being reported so I'll make sure I do that every time I'll be coming across something strange, in what NABPS' content sources are concerned.
Further on, I have added Tudor Damian's feed to the list as I found its contents as being of interest to the community.
I hope this helps everyone involved,
@Dexter
Please, kindly be notified that this feed aggregator is only able to process RSS2.0 and ATOM feeds that are, XML-wise, well formed. If your feed is not well formed, it will not pass into the CFF cache and, subsequently, neither into the SFS cache, to the output drivers and in the final output (be that HTML, RSS or ATOM). The net effect of this is that your content will not show up in the aggregator, even though NABPS will fetch it as dictated by If-Modified-Since and Last-Modified headers.
Please note that the two most common causes encountered for an otherwise working feed going not well formed are:
- lack of cooperation between the content generator (blog software) and the web server, in effect making the web server believe the feed is updated more or less often (or not at all) than it actually is; in some cases NABPS has even been fed empty or truncated feeds due to the above
- lack of cooperation between the templating engine used on the generating site and the strict nature of XML dialects (most sites generate their feeds from a custom template, processed the same way as the HTML-generating ones) -- many if not all of the things that are allowed or even acceptable in HTML are not tolerated in XML so please check your templates and, most importantly, your feed (i.e. the end result of said processing). Over time, NABPS has been fed countless cases of feeds containing PHP debugging output (or even warnings or errors) interspersed with XML, not to mention the almost everyday case when an otherwise well-formed feed comes in with trailing garbage of the aforementioned kind.
Anticipating the FAQ: no I shall not attempt to relax NABPS' XML parser in any way that would make it susceptible of accepting a not well-formed feed as valid. Standards must be obeyed, to the letter.
One, mandatory, example: Alexandru Burlacu's feed ends with the following (closing rss tag shown for clarity):
</rss>(later edit, on the same day, at 15:02EEST)
<br />
<b>Warning</b>: preg_match() [<a href='function.preg-match'>function.preg-match</a>]: Compilation failed: nothing to repeat at offset 14 in <b>/var/www/alex.burlacu.blogsite.org/blog/wp-content/plugins/wassup/wassup.php</b> on line <b>2782</b><br />
It seems the above problem has been fixed shortly after my post -- thank you for your timely reaction. This is not the first time things get fixed after being reported so I'll make sure I do that every time I'll be coming across something strange, in what NABPS' content sources are concerned.
Further on, I have added Tudor Damian's feed to the list as I found its contents as being of interest to the community.
I hope this helps everyone involved,
@Dexter
Hello everyone,
This is just a short note to let you know I've written a planet info block driver for NABPS and as such you can now see its status summary on the front page.
The code audit in what XML parsing semantics are concerned (i.e. making sure my code interprets the source feeds exactly the way the standard DTDs meant it) is due this Wednesday night; packing, signing, legal clean-up and documenting will probably take another evening or so (at the current load figure) so all in all I expect to release NABPS v1.0 somewhere during this Friday, probably Friday evening :-)
(later edit at 00:11EET on Tuesday, February 3rd, 2009)
Mr. Alexandru Popescu's blog feed has just lent me a very nice example of a "thing that needs fixing in NABPS" as that particular feed is output as a very strange case of ATOM-in-RSS2.0 which, even if well formed and well documented in the appropriate DTDs, is not handled properly by NABPS resulting in a post entry with, apparently, no content.
I've fixed the problem temporarily by switching to the native-ATOM version of the above and will be investigating the issue to its full extent (and, hopefully, resolution) during the XML code audit.
Apart from that, upcoming events include the opening of a new production studio for Radio Andromeda and the launch of my personal site (as I am now self-employed).
Also, a new issue of Since you have asked is almost due as I've received some very interesting questions over the past few weeks -- will probably be posting it on Monday evening.
That's all for now,
Stay tuned,
@Dexter
This is just a short note to let you know I've written a planet info block driver for NABPS and as such you can now see its status summary on the front page.
The code audit in what XML parsing semantics are concerned (i.e. making sure my code interprets the source feeds exactly the way the standard DTDs meant it) is due this Wednesday night; packing, signing, legal clean-up and documenting will probably take another evening or so (at the current load figure) so all in all I expect to release NABPS v1.0 somewhere during this Friday, probably Friday evening :-)
(later edit at 00:11EET on Tuesday, February 3rd, 2009)
Mr. Alexandru Popescu's blog feed has just lent me a very nice example of a "thing that needs fixing in NABPS" as that particular feed is output as a very strange case of ATOM-in-RSS2.0 which, even if well formed and well documented in the appropriate DTDs, is not handled properly by NABPS resulting in a post entry with, apparently, no content.
I've fixed the problem temporarily by switching to the native-ATOM version of the above and will be investigating the issue to its full extent (and, hopefully, resolution) during the XML code audit.
Apart from that, upcoming events include the opening of a new production studio for Radio Andromeda and the launch of my personal site (as I am now self-employed).
Also, a new issue of Since you have asked is almost due as I've received some very interesting questions over the past few weeks -- will probably be posting it on Monday evening.
That's all for now,
Stay tuned,
@Dexter
Hello,
NABPS got a small cosmetic fix in what the feed list is concerned (see SVN): the individual blogs are now listed first, followed by the group/company blogs and they are all sorted alphabetically ascending.
Moreover, I have ressurected diacritical marks on all authors' names -- if I have maimed anyone's name, please accept my apologies and let me know which one so I can fix it right away.
If no ugly bugs surface today, I shall replace Planet with NABPS tonight.
That's all for now,
@Dexter
NABPS got a small cosmetic fix in what the feed list is concerned (see SVN): the individual blogs are now listed first, followed by the group/company blogs and they are all sorted alphabetically ascending.
Moreover, I have ressurected diacritical marks on all authors' names -- if I have maimed anyone's name, please accept my apologies and let me know which one so I can fix it right away.
If no ugly bugs surface today, I shall replace Planet with NABPS tonight.
That's all for now,
@Dexter
Hello everyone,
This is just a short note to update you all on two facts:
That's all for now,
@Dexter
This is just a short note to update you all on two facts:
- NABPS v0.5-alpha got its RSS 2.0 input driver fixed and as a result of that rendering feeds of said format is now more robust -- which is incidentaly why you can now actually see IT-ist's posts in the aggregated output :-)
- I have started a Romanian language replica to this blog, in the hope that I will be able to get to more of you by means of that.
That's all for now,
@Dexter
This is just a short note to update you all on the progress with NABPS, ChangeLog-style:
On a rather unrelated level, I have also modified my feed templates (i.e. on this blog) to output more features, including a feed icon -- much to the enjoyment (hopefully) of my audience as many of them do not have a hint about my appearance ;-)
That's about it for now,
The next thing to do is implement the concept of a Feed Name (!= Feed Title) and always sort the Feed Roll alphabetically ascending by Feed Name when output.
Keep those bug reports coming,
@Dexter
- OPML and FOAF output added, operational and tested
- New feed icons, red is for entry feeds, blue is for feed rolls
- Made the entry icon (inherited from its corresponding feed) part of the link pointing to the blog that entry came from
- Fixed a bug in what the caching discipline was concerned: if the Feed List was modified but no feeds were found modified (i.e. HTTP If-Modified-Since), the cache digest would not get written out to disk and, as a result, the modifications just made to the Feed List would not propagate into the system
- Various template-related fixes to achieve standards compliance
- Other minor fixes, see the Subversion repository logs.
On a rather unrelated level, I have also modified my feed templates (i.e. on this blog) to output more features, including a feed icon -- much to the enjoyment (hopefully) of my audience as many of them do not have a hint about my appearance ;-)
That's about it for now,
The next thing to do is implement the concept of a Feed Name (!= Feed Title) and always sort the Feed Roll alphabetically ascending by Feed Name when output.
Keep those bug reports coming,
@Dexter
Good evening to those who will go to sleep soon and good morning to those who just woke up :-)
It is with great pleasure that I announce the launch of the first version of NABPS, namely 0.5-alpha. This will change into 0.5-beta as soon as all the extra features are in (see below) and, subsequently, into 0.75-rc1 as soon as common-sense testing takes place and is passed with flying colours.
NABPS is an acronym for New Aggregator Because Planet Sucks and is meant as a reminder for the main reason of starting to develop it, in the first place.
You can see a live demo here (you can always compare it with the original here).
In an attempt to make NABPS compatible with RFC-ignorant web servers (RFC2616, section 14.3 comes to mind), I have tweaked some options I'm feeding to CURL in the Fetcher which seems to have fixed the "Oops, is that gzip? I was expecting XML!" problem.
Unfortunately, this still does not guard against uninspired WordPress plugins, such as WP Super Cache, which (even if transparently decompressed) do not produce well-formed XML, as you can see in the example below:
Unfortunately for its owner, the feed above has been disabled until such time it shall be fixed.
Furthermore, while I am still adding features and fixing obvious bugs there will be no "release" package so, to compensate for that, here is the Subversion repository if you wish to check the code out (and here is the DAV version of the above, for direct access with svn, TortoiseSVN and the like).
Finally, I did not mention (and the sources do not, as of yet, reflect that), but all code is released under the GPL, version 2.
Please tell me what you think (obviously, if feedback is positive NABPS will replace Planet at the above address -- and fast at that).
Have a nice week ahead and don't forget to leave your feedback here (if I am not told what does not work, I cannot fix it :-) ),
@Dexter
It is with great pleasure that I announce the launch of the first version of NABPS, namely 0.5-alpha. This will change into 0.5-beta as soon as all the extra features are in (see below) and, subsequently, into 0.75-rc1 as soon as common-sense testing takes place and is passed with flying colours.
NABPS is an acronym for New Aggregator Because Planet Sucks and is meant as a reminder for the main reason of starting to develop it, in the first place.
You can see a live demo here (you can always compare it with the original here).
In an attempt to make NABPS compatible with RFC-ignorant web servers (RFC2616, section 14.3 comes to mind), I have tweaked some options I'm feeding to CURL in the Fetcher which seems to have fixed the "Oops, is that gzip? I was expecting XML!" problem.
Unfortunately, this still does not guard against uninspired WordPress plugins, such as WP Super Cache, which (even if transparently decompressed) do not produce well-formed XML, as you can see in the example below:
<!-- Page not cached by WP Super Cache. No closing HTML tag. Check your theme. -->
<?xml version="1.0" encoding="UTF-8"?>Unfortunately for its owner, the feed above has been disabled until such time it shall be fixed.
Furthermore, while I am still adding features and fixing obvious bugs there will be no "release" package so, to compensate for that, here is the Subversion repository if you wish to check the code out (and here is the DAV version of the above, for direct access with svn, TortoiseSVN and the like).
Finally, I did not mention (and the sources do not, as of yet, reflect that), but all code is released under the GPL, version 2.
Please tell me what you think (obviously, if feedback is positive NABPS will replace Planet at the above address -- and fast at that).
Have a nice week ahead and don't forget to leave your feedback here (if I am not told what does not work, I cannot fix it :-) ),
@Dexter
Today's topic in this category: Internet Protocol, Version 6. Pure perfection but, sadly, no one cares :-(
Just compare, this (using IPv6):
with this (using IPv4):
And you tell me which you like better!
@Dexter
Just compare, this (using IPv6):
# traceroute6 download.wikimedia.org traceroute to download.wikimedia.org (2620:0:860:2:230:48ff:fe5a:eb1e), 30 hops max, 40 byte packets 1 atlas.linux360.ro (2001:5c0:9a05::1) 0.285 ms 0.279 ms 0.279 ms 2 2001:5c0:8fff:fffe::79a4 (2001:5c0:8fff:fffe::79a4) 147.944 ms 149.277 ms 150.772 ms 3 * * * 4 if-5-0-1.6bb1.mtt-montreal.ipv6.teleglobe.net (2001:5a0:300::5) 193.968 ms 195.987 ms 201.568 ms 5 if-1-0.mcore3.mtt-montreal.ipv6.teleglobe.net (2001:5a0:300:100::1) 195.907 ms 195.897 ms 200.469 ms 6 if-13-0.mcore4.nqt-newyork.ipv6.teleglobe.net (2001:5a0:300:100::2) 206.977 ms 206.969 ms 206.930 ms 7 if-4-0.mcore4.pdi-paloalto.ipv6.teleglobe.net (2001:5a0:1200::1) 275.155 ms 267.837 ms 267.607 ms 8 2001:5a0:1200::6 (2001:5a0:1200::6) 267.070 ms * * 9 2001:5a0:1200:100::e (2001:5a0:1200:100::e) 266.977 ms * * 10 (2610:18:10a::2) 346.465 ms 346.458 ms 346.426 ms 11 (2610:18:10a::2) 347.111 ms 347.096 ms 342.075 ms 12 2620:0:860:2:230:48ff:fe5a:eb1e (2620:0:860:2:230:48ff:fe5a:eb1e) 342.051 ms 341.723 ms 341.700 ms
with this (using IPv4):
# traceroute download.wikimedia.org traceroute to download.wikimedia.org (208.80.152.183), 30 hops max, 40 byte packets 1 gw.dexter.linux360.ro (192.168.10.1) 0.396 ms 0.384 ms 0.395 ms 2 (89.37.42.1) 1.096 ms 1.523 ms 1.754 ms 3 79.134.34.217 (79.134.34.217) 1.200 ms 1.661 ms 1.862 ms 4 79.134.32.137 (79.134.32.137) 1.160 ms 1.102 ms 1.852 ms 5 rc1-buc-uplink2-rd1-buc.airbites.net (195.170.181.169) 1.525 ms 0.981 ms 1.414 ms 6 GigabitEthernet2-1-106.ipcolo2.frankfurt.level3.net (62.67.38.185) 219.299 ms 219.417 ms 219.401 ms 7 vlan89.csw3.Frankfurt1.Level3.net (4.68.23.190) 46.631 ms 46.562 ms 46.577 ms 8 ae-81-81.ebr1.Frankfurt1.Level3.net (4.69.140.9) 59.094 ms 58.886 ms 59.033 ms 9 ae-2.ebr2.Paris1.Level3.net (4.69.132.141) 65.521 ms 65.465 ms 65.631 ms 10 ae-41.ebr2.Washington1.Level3.net (4.69.137.50) 149.641 ms ae-43.ebr2.Washington1.Level3.net (4.69.137.58) 149.121 ms ae-42.ebr2.Washington1.Level3.net (4.69.137.54) 147.043 ms 11 ae-62-62.csw1.Washington1.Level3.net (4.69.134.146) 139.431 ms 136.973 ms ae-72-72.csw2.Washington1.Level3.net (4.69.134.150) 137.448 ms 12 ae-81-81.ebr1.Washington1.Level3.net (4.69.134.137) 137.381 ms ae-91-91.ebr1.Washington1.Level3.net (4.69.134.141) 141.602 ms ae-61-61.ebr1.Washington1.Level3.net (4.69.134.129) 141.883 ms 13 ae-2.ebr3.Atlanta2.Level3.net (4.69.132.85) 152.735 ms 165.104 ms 165.049 ms 14 ae-62-60.ebr2.Atlanta2.Level3.net (4.69.138.3) 157.452 ms ae-72-70.ebr2.Atlanta2.Level3.net (4.69.138.19) 153.395 ms ae-62-60.ebr2.Atlanta2.Level3.net (4.69.138.3) 162.608 ms 15 ae-1-6.bar1.Tampa1.Level3.net (4.69.137.113) 170.868 ms 170.782 ms 170.258 ms 16 ae-5-5.car1.Tampa1.Level3.net (4.69.133.49) 171.663 ms 171.380 ms 171.518 ms 17 ae-13-13.car3.Tampa1.Level3.net (4.69.133.54) 170.908 ms 171.654 ms 171.454 ms 18 e1-11-level3.co2.tpax.as30217.net (4.71.0.10) 170.310 ms 169.683 ms 169.681 ms 19 ge8-1.csw5-pmtpa.wikimedia.org (66.113.197.94) 170.530 ms 170.073 ms 170.015 ms 20 storage2.wikimedia.org (208.80.152.183) 170.670 ms 171.357 ms 169.819 ms
And you tell me which you like better!
@Dexter
Following Monday evening's test (with the usual live show session) and thanks to some uniquely useful input from d3vi1 and his friend Mircea, I have come to the following configuration for the Marconi Transmitter:
As always, trouble never shows up alone, so here they are:
Good night,
@Dexter
#!/usr/bin/liquidsoap
# Import system-wide settings
%include "/etc/liquidsoap/settings.liq"
# Fallback/Background sound sources
playlist_monday = playlist(id="monpls", mode="random", "/usr/local/etc/liquidsoap/monday.m3u")
playlist_tuesday = playlist(id="tuepls", mode="random", "/usr/local/etc/liquidsoap/tuesday.m3u")
playlist_wednesday = playlist(id="wedpls", mode="random", "/usr/local/etc/liquidsoap/wednesday.m3u")
playlist_thursday = playlist(id="thupls", mode="random", "/usr/local/etc/liquidsoap/thursday.m3u")
playlist_friday = playlist(id="fripls", mode="random", "/usr/local/etc/liquidsoap/friday.m3u")
playlist_weekend = playlist(id="wepls", mode="random", "/usr/local/etc/liquidsoap/weekend.m3u")
# Add them up
background = switch([({1w}, playlist_monday),
({2w}, playlist_tuesday),
({3w}, playlist_wednesday),
({4w}, playlist_thursday),
({5w}, playlist_friday),
({6w or 7w}, playlist_weekend)])
# Leave some room for interactivity (Public Requests)
backandreq = fallback([request.equeue(id="pubreq"),
background])
# Leave some room for management (External status callbacks)
def liveup()
system("/usr/share/icecast/scripts/live-up.sh")
end
def livedown()
system("/usr/share/icecast/scripts/live-down.sh")
end
# Plug the live studio in ...
production = input.harbor(id="production", on_connect=liveup, on_disconnect=livedown, "live.ogg")
# ... and make sure it is heard when On Air
master_out = fallback(track_sensitive=false,
[production,
backandreq])
# Leave room for some more interactivity (RDS Ticker)
masterandrds = insert_metadata(id="rds", master_out)
# Emulate the final stage before the Tx PA: dynamics processing
def tx_dsp(s)
# Emulate a three-band crossover network
low = filter.iir.eq.low(frequency=200.)
mhi = filter.iir.eq.high(frequency=140.)
mlo = filter.iir.eq.low(frequency=2500.)
hhi = filter.iir.eq.high(frequency=2400.)
hlo = filter.iir.eq.low(frequency=15000.)
comp = compress(rms_window=0.03)
# Normalize (perform AGC)
n = normalize(target=0., window=0.03, gain_min=-10., gain_max=10., s)
# Emulate a three-band dynamic range compressor
c = add(normalize=false,
[comp(attack=1., release=50., threshold=-27., ratio=5., gain=-4., knee=10., low(n)),
comp(attack=1., release=80., threshold=-17., ratio=3., gain=-4., knee=10., mlo(mhi(n))),
comp(attack=1., release=80., threshold=-20., ratio=7., gain=-4., knee=10., hlo(hhi(n)))])
# Limit the signal output to prevent clipping
limit(attack=1., release=100., threshold=-2., rms_window=0.001, gain=0., c)
end
tx_out = tx_dsp(mksafe(masterandrds))
# And, finally, send it all to the transmitter
output.icecast.mp3(host="localhost", password="", bitrate=96, samplerate=44100, mount="live",
description="YO3HVT Radio", genre="Talk Radio", url="http://radio.dexter.linux360.ro/",
tx_out)
output.icecast.vorbis(host="localhost", password="", quality=-0.5, samplerate=44100, mount="live.ogg",
description="YO3HVT Radio", genre="Talk Radio", url="http://radio.dexter.linux360.ro/",
tx_out)
As always, trouble never shows up alone, so here they are:
- the Intel Pentium running @ 1GHz inside Atlas just cannot cope with applying a 4th order Butterworth filter in realtime over a 44.1kHz stereo signal coming in 16bit-wide samples :-( -- so I'm stuck with IIR filters and making up a bandpass version of those by connecting two of the same in series
- if the weekend configuration drove Atlas to ~0.85 loadavg (and 90% CPU usage); tonight configuration (albeit, sounding a lot better) pushes it to 1.01 loadavg and 95% CPU usage -- I think that's about as much as I can push it before making a mess of the whole thing :(
- connect the dynamics processor to the Production room input only, such that it will only be used by live shows (leaving the CPU less tired during non-live activity for, e.g. system jobs) -- easy on CPU, bad on off-peak hour quality :(
- buy more hardware, specifically a node dedicated to the whole sound processing/enhancing job (it's easy to link it to other liquidsoap instances via the input.rtp() and output.rtp() operators) -- good on quality, bad on the electricity bill and my wallet
- move some of the processing to the Production room -- defeats initial design aim, probably kills the CPU power headroom I found very useful to have at hand in a few critical situations while on air
Good night,
@Dexter
Most of you know from Friday's show (sorry to those who missed it, no tape, it was an unplugged performance :( ) that I've made major changes in what the digital (or purely-software-based) part of Radio Andromeda's Marconi studio is concerned.
My initial intent was to separate batch processing duties from the live ones and distribute the former to Atlas and the latter to BeatrixKiddo -- in effect splitting the Marconi location between a Transmitter and a Production room.
As is always the case with science: easier said than done! I used Savonet's liquidsoap for the Transmitter (alongside IceCast2 as a final Tx PA and antenna, of course) and I stuck to the hybrid configuration made up of XMMS, linphone and darkice on the digital side and Behringer's UCA200, UCA202, XENIX1002, C-3, XM8500, HA400 and HPM1000 on the analog one for the Production.
My design intent was to identify all processing (be that of the audio signal or otherwise) that would not change during normal operations of the station and move it completely out of the way, on the Transmitter -- leaving ample room for experiments (trial and error) and flexibility in Production.
Identifying was the easy job: the Tx PA is constant and so is the relay network; the background (i.e. when not live) music is again static; finally, any generic signal processing (i.e. enhancements) is again static and therefore belongs on the Transmitter.
Implementation was not so :( Moving the entire music collection within Atlas' reach was rather easy compared to building playlists for each day out of nearly 5000 tracks. Then came the problems: liquidsoap does not (yet) know how to read FLAC, nor MPEG 4 Audio (AAC+) so those tracks had to be grep -v -ed out of the daily playlists. Furthermore, a means had to be devised to connect Production to the output stream when the former would be airing -- fortunately the track-insensitive version of the fallback() operator in liquidsoap did the job nicely.
But the most mind boggling task of all was the dynamics processor + AGC. Thanks to the excellent documentation in liquidsoap and to the great freeverb3 project (and its XMMS plugin, as a working example), I was finally able to arrive at a working configuration. Whether the result is satisfactory, it's up to you to decide, my dear listeners :-)
Anyway, here is the code that controlls it all (the Transmitter part):
What's next, you might ask? Well, the web interface for submitting those requests that the code above can serve ;-) and many many more to come. Stay tuned,
Have a nice week,
@Dexter
My initial intent was to separate batch processing duties from the live ones and distribute the former to Atlas and the latter to BeatrixKiddo -- in effect splitting the Marconi location between a Transmitter and a Production room.
As is always the case with science: easier said than done! I used Savonet's liquidsoap for the Transmitter (alongside IceCast2 as a final Tx PA and antenna, of course) and I stuck to the hybrid configuration made up of XMMS, linphone and darkice on the digital side and Behringer's UCA200, UCA202, XENIX1002, C-3, XM8500, HA400 and HPM1000 on the analog one for the Production.
My design intent was to identify all processing (be that of the audio signal or otherwise) that would not change during normal operations of the station and move it completely out of the way, on the Transmitter -- leaving ample room for experiments (trial and error) and flexibility in Production.
Identifying was the easy job: the Tx PA is constant and so is the relay network; the background (i.e. when not live) music is again static; finally, any generic signal processing (i.e. enhancements) is again static and therefore belongs on the Transmitter.
Implementation was not so :( Moving the entire music collection within Atlas' reach was rather easy compared to building playlists for each day out of nearly 5000 tracks. Then came the problems: liquidsoap does not (yet) know how to read FLAC, nor MPEG 4 Audio (AAC+) so those tracks had to be grep -v -ed out of the daily playlists. Furthermore, a means had to be devised to connect Production to the output stream when the former would be airing -- fortunately the track-insensitive version of the fallback() operator in liquidsoap did the job nicely.
But the most mind boggling task of all was the dynamics processor + AGC. Thanks to the excellent documentation in liquidsoap and to the great freeverb3 project (and its XMMS plugin, as a working example), I was finally able to arrive at a working configuration. Whether the result is satisfactory, it's up to you to decide, my dear listeners :-)
Anyway, here is the code that controlls it all (the Transmitter part):
#!/usr/bin/liquidsoap
# Import system-wide settings
%include "/etc/liquidsoap/settings.liq"
# Fallback/Background sound sources
playlist_monday = playlist(id="monpls", mode="random", "/usr/local/etc/liquidsoap/monday.m3u")
playlist_tuesday = playlist(id="tuepls", mode="random", "/usr/local/etc/liquidsoap/tuesday.m3u")
playlist_wednesday = playlist(id="wedpls", mode="random", "/usr/local/etc/liquidsoap/wednesday.m3u")
playlist_thursday = playlist(id="thupls", mode="random", "/usr/local/etc/liquidsoap/thursday.m3u")
playlist_friday = playlist(id="fripls", mode="random", "/usr/local/etc/liquidsoap/friday.m3u")
playlist_weekend = playlist(id="wepls", mode="random", "/usr/local/etc/liquidsoap/weekend.m3u")
# Add them up
background = switch([({1w}, playlist_monday),
({2w}, playlist_tuesday),
({3w}, playlist_wednesday),
({4w}, playlist_thursday),
({5w}, playlist_friday),
({6w or 7w}, playlist_weekend)])
# Leave some room for interactivity (Public Requests)
backandreq = fallback([request.equeue(id="pubreq"),
background])
# Leave some room for management (External status callbacks)
def liveup()
system("/usr/share/icecast/scripts/live-up.sh")
end
def livedown()
system("/usr/share/icecast/scripts/live-down.sh")
end
# Plug the live studio in ...
production = input.harbor(id="production", on_connect=liveup, on_disconnect=livedown, "live.ogg")
# ... and make sure it is heard when On Air
master_out = fallback(track_sensitive=false,
[production,
backandreq])
# Leave room for some more interactivity (RDS Ticker)
masterandrds = insert_metadata(id="rds", master_out)
# Emulate the final stage before the Tx PA: dynamics processing
def tx_dsp(s)
# Emulate a three-band crossover network
low = filter.iir.eq.low(frequency=140.)
mhihalf = filter.iir.eq.high(frequency=130.)
mlohalf = filter.iir.eq.low(frequency=6400.)
high = filter.iir.eq.high(frequency=6300.)
comp = compress(rms_window=0.03)
# Emulate a three-band dynamic range compressor
c = add(normalize=false,
[comp(attack=1., release=50., threshold=-27., ratio=5., gain=-4., knee=10., low(s)),
comp(attack=1., release=80., threshold=-17., ratio=3., gain=-4., knee=10., mlohalf(mhihalf(s))),
comp(attack=1., release=80., threshold=-20., ratio=7., gain=-4., knee=10., high(s))])
# Normalize (perform AGC) and then limit the signal
limit(attack=1., release=100., threshold=-2., rms_window=0.001, gain=0.,
normalize(target=0., window=0.03, gain_min=-10., gain_max=10., c))
end
tx_out = tx_dsp(mksafe(masterandrds))
# And, finally, send it all to the transmitter
output.icecast.mp3(host="localhost", password="", bitrate=64, samplerate=44100, mount="live",
description="YO3HVT Radio", genre="Talk Radio", url="http://radio.dexter.linux360.ro/",
tx_out)
output.icecast.vorbis(host="localhost", password="", quality=-0.5, samplerate=44100, mount="live.ogg",
description="YO3HVT Radio", genre="Talk Radio", url="http://radio.dexter.linux360.ro/",
tx_out)
What's next, you might ask? Well, the web interface for submitting those requests that the code above can serve ;-) and many many more to come. Stay tuned,
Have a nice week,
@Dexter
Most entities nowadays seem to constantly perform better if moved (i.e. morphed) away from their creator(s) intent and design -- and I say that with a bitter note, engineering (A/V, IT, Tc etc. in particular) should be a very precise and predictable field where each step is calculated not only in what how is it going to be taken is concerned but rather in what what will the consequences be is. Sad ...
After many years of dreaming about it and three years after I first touched such a thing (I had to setup and fill one up for one of my highest-ranking bosses at that time), I got my hands on a 5th generation (improved) Apple iPod with Video (the black, 80GB version). Why? Because it's still the only portable music player with such a big hard drive -- I do not have access to the Japanese consumer market, mind you ;-) Yet ...
No, I would not call the feeling "love at first sight" -- that was years ago and by present times it'd already gotten stale (pretty much the same thing that happens with humans and feelings left unexplored/dreams that never came true) but I was pleasantly impressed, at least with the hardware (eye candy) part. Apple is renowned for "nice" (in a rather photographic way) products and the Video iPod in front of me proudly confirmed it. Unfortunately (for Apple), that's about all the good things that can be said about their iPod as a whole product: good looking, period.
Naturally, after cleaning it (it's a second hand issue), I thought I'd put all my music on it (about 18GB at present in some ~3800 files) and start listening, but Apple had many a surprise in store for me. In no particular order, here they are:
As with any open source thing (not necessarily Linux and RockBox is not Linux for example) I've ever laid my hands upon, quality, efficiency and a simple and clear design strike you the second you open the box/archive etc.
Installation is a 5 minute job, divided between getting the actual software on the iPod (just unzip a few .ZIP files in its root) and changing the boot loader (just run a simple program that does all the work for you). Extra GUI-art (fonts, skins etc.) is one download-and-unzip away.
After installation, you just reboot the device and there you go: Apple iPod running RockBox :-) Of course (if you were asking yourselves), the installation is non-destructive: you can always choose to reboot in the original Apple firmware -- but who would want that when RockBox is so much better than that crap? :-)
So, to sum it up in a few lines:
That about sums it up: if you get an iPod (less for the latest one, the "Classic" which has an encrypted firmware -- I really wish they'd rot for that) and want to actually use it, you should really give RockBox a try!
Good night everyone,
@Dexter
After many years of dreaming about it and three years after I first touched such a thing (I had to setup and fill one up for one of my highest-ranking bosses at that time), I got my hands on a 5th generation (improved) Apple iPod with Video (the black, 80GB version). Why? Because it's still the only portable music player with such a big hard drive -- I do not have access to the Japanese consumer market, mind you ;-) Yet ...
No, I would not call the feeling "love at first sight" -- that was years ago and by present times it'd already gotten stale (pretty much the same thing that happens with humans and feelings left unexplored/dreams that never came true) but I was pleasantly impressed, at least with the hardware (eye candy) part. Apple is renowned for "nice" (in a rather photographic way) products and the Video iPod in front of me proudly confirmed it. Unfortunately (for Apple), that's about all the good things that can be said about their iPod as a whole product: good looking, period.
Naturally, after cleaning it (it's a second hand issue), I thought I'd put all my music on it (about 18GB at present in some ~3800 files) and start listening, but Apple had many a surprise in store for me. In no particular order, here they are:
- iTunes is a total pain in the rear. Huge download, ugly design (Windows -guidelines -wise!), bloated like hell (why would I need an extra music player? etc.), hard to control (seems like in Apple world concepts like "net-wise privacy", "slow connections", "off line mode", "pause current operation" etc. are totally unknown)
- it was absolutely impossible to use Apple software to perform a simple, atomic copy of one folder (containing music data) from my computer to the iPod (and have that data playable on the iPod afterwards)
- iTunes attempted to retrieve album art for no less than 2210 albums, most of which were named "Unknown Album", "new", "" (!) and "Album 1" -- without asking first whether or not I desire to access the Internet for that and without telling me what data exactly is being sent out. All that resulted in no usable album art being retrieved, 20MiB of HTTP 404 traffic (!) and 15 minutes of my precious time being lost
- iTunes installs QuickTime without letting me have a say (and the same applies for a couple of other obscure services that were added to my system and started without me asking for it). My use case is just copying music to my device -- no decoding or even playback -- so why should any of QuickTime be involved?
- iTunes barfed on the first of my few .WMA files and said it will convert it to .AAC because the iPod can't play it as is. Need I remind you this is the Windows iTunes (and a FAT32 -formatted iPod) version?! I don't think Bill Gates will be very happy to hear that his mind child audio format is "not playable" on a device made by a company on his own payroll ...
- iTunes totally ignored my .OGG and .FLAC files -- this is a bit fascist, don't you think? What we cannot understand, does not exist! I took great offense in that behavior ...
- iTunes renames and hides my music when "syncing" it to the iPod, leaving me with an ID3 Tag-based browsing view only -- of course, without asking whether I am ok with having my files maimed. I once again took great offense in that, especially when it comes from a company that claims the trend setter role in everything it does
As with any open source thing (not necessarily Linux and RockBox is not Linux for example) I've ever laid my hands upon, quality, efficiency and a simple and clear design strike you the second you open the box/archive etc.
Installation is a 5 minute job, divided between getting the actual software on the iPod (just unzip a few .ZIP files in its root) and changing the boot loader (just run a simple program that does all the work for you). Extra GUI-art (fonts, skins etc.) is one download-and-unzip away.
After installation, you just reboot the device and there you go: Apple iPod running RockBox :-) Of course (if you were asking yourselves), the installation is non-destructive: you can always choose to reboot in the original Apple firmware -- but who would want that when RockBox is so much better than that crap? :-)
So, to sum it up in a few lines:
- .MP3, .WMA, .WAV, .AAC, .OGG, .FLAC, .APE (and the list could go on), all supported and played back with no problem
- real graphic and parametric equalizer, fully tunable to your taste and including saving and loading presets
- fully skin-able GUI: just grab your preferred editor (including the one included in RockBox) and write a new skin :-)
- many many other nice goodies you would expect from an open source release ... hmm, if I'll tell you I can actually play Doom on it, would it be enough? :-)
- both ID3-based and directory-based music access
- powerful playlist management, including .M3U format support and on-the-fly playlist creation and saving
That about sums it up: if you get an iPod (less for the latest one, the "Classic" which has an encrypted firmware -- I really wish they'd rot for that) and want to actually use it, you should really give RockBox a try!
Good night everyone,
@Dexter
