I've just uploaded Smack 4.1.8 and 4.2.0-beta2 to Maven Central.
Smack 4.1.8 fixes a few minor issues and is expected to be the last release of the 4.1 branch. As always, Smack releases with the same major and minor version numbers are drop in replacements. Ideally you just need to change a single variable somewhere in your build system.
Smack 4.2.0-beta2 is the latest beta of Smack's current development branch. Notable additions include support for XEP-0313: Message Archive Management (MAM) and the IoT XEP series.
We are announcing today MUC Subscription, a new experimental feature in ejabberd that make group conversations in XMPP work seamlessly with mobile clients.
Today, a large proportion of XMPP chat projects are targeting both mobile, focusing on Whatsapp-like group conversations. Developers of such projects had been deeply frustrated because it was not possible to easily implement this feature on top of XMPP group chat features.
The main issue with XMPP group chat is that it is bound to user presence. Group chat in XMPP is based on XEP-0045: Multi-User Chat. It was designed to replicate IRC-like chat room, at a time where mobile internet connections were very rare. As a result, the protocol is based on presence. You join a room when you want to participate. You leave it when you are done.
On mobile, presence is almost irrelevant. You are always somewhat loosely reachable. However, your application may not run all the time because, to save battery life, mobile operating systems will hibernate your application when you are not actively using it. Mobile applications are supposed to be reachable through push notification services as they have generally no open TCP connection to the server in background state.
From XMPP point of view, it means that when the application is asleep, the XMPP client is not online. When it is not online, the user cannot be a participant in any chat room. He does not get messages nor notifications and thus misses events in the chat room.
Moreover, every time the user gets online, he has to rejoin all his ongoing conversations (MUC rooms). This is very consuming in terms of battery life and bandwidth.
Several workarounds had been tried by developers:
However, none of this approach did prove to be an easy straightforward solution to the problem. It did not work very well in the sense that these work-around are not totally transparent and affect several parts of your client development.MIX to the rescue
There is a work-in-progress at the XMPP Standard Foundation (XSF) that aims to build a full rework of the Multi-User Chat specification, taking the requirement to avoid relying on presence into account.
The specification is called XEP-0369: Mediated Information eXchange (MIX). ejabberd already implements the draft since a few months and we are actively working on it and supported the effort.
However, this is a work-in-progress that will take time before being fully usable as a Multi-User Chat replacement.
As we wanted to solve the problem today, for all XMPP developers, we decided to explore a path in-between. We developed a smaller, less ambitious extension, that solves the issues developers are facing to implement mobile group chat today. We are still committed to MIX implementation and we think our proposed extension will please mobile developers and help us channel feedback to the XSF to build future proof MIX specification.Introducing MUC Sub
So, what is MUC Sub about and how is it designed ?
The goal of MUC Sub is to try to rely as much as possible on existing MUC specification, while making the smallest possible change that make mobile group conversation client easy.
We found that we only missed a few components. To play well with mobile, we just need to add the ability to send and receive messages to a room without having to send presence to the room. More generally we have to allow other type of interactions with the room (like configuration changes for example or kick and ban) without having to send presence to it to become a participant.
Basically, you need to be able to have a permanent subscription to a given room to receive the events. That subscription should grant you the right to send message and interact with the room. The name of our extension come from that simple idea: Add subscription feature to Multi-User Chat, hence MUC Sub.
We found we can put all the pieces together by reusing existing XMPP protocol components:
After writing a draft specification and a working implementation, we are very happy with the result: We have a working MUC Sub implementation that enables mobile group conversations. And we did all this by leveraging existing MUC specification and staying compliant with it.
The draft protocol is published on ejabberd documentation site: MUC Sub Draft
ejabberd main repository contains the code for MUC Sub. You can find it on Github: ejabberd.
The feature is enabled by default. To use it, just make sure you set the new parameter “Allow subscription” in the room on which you want to use it.Next steps
We suggest, you can read the draft document we wrote, use the implementation and develop a mobile client.
Please, while you do so, send us your feedback about your success and troubles implementing your mobile group conversations.
MUC Sub is already available in ejabberd Community Edition repository and is going to be available as an experimental feature in ejabberd 16.07.
It will also be deployed as an experimental feature for ejabberd SaaS platform during the Summer. We are planning to test it as soon as possible with existing customers that are already willing to use it in production.
We hope you will like the approach and develop great new mobile clients thanks to ejabberd and MUC Sub !
In this video, Mickaël Rémond explains the features, options and API that can be used to protect ejabberd servers from spam and various other types of attacks.
The video was recorded at ejabberd Advanced Erlang Workshop in Paris.
You can watch the full talk online:
You can also download the slides: Fighting XMPP abuse and spam with ejabberd.
I spent the day learning more about how tvOS and the focus engine work. I’ve further improved the UI for TVPortal. I think the details screen is final, the recordings screen is almost done and playback screen is done. This has been an interesting project and quite educational. I haven’t really dabbled in media streaming in a while.
Starting with b91978dcc4ae partial support for the IoT XEPs was added to Smack. The XEPs consists, amongst other XEPs, of
The XEPs are in experimental state, which means changes to them are possible.
Smack does currently only support a partial set of the mechanisms specified, especially when it comes to Data and Control. For example only boolean and integer values can be read and written But support for more data types can be easily added.
The development of the API was sponsored by Clayster.
Clayster creates technology to secure trust in the transactions between physical and digital entities, and strives to be that generic foundation for your physical assets digital life.
Clayster has an IoT discovery and provisioning platform supporting XEP-0347 and XEP-0324. The platform is available for those who are interested to explore XMPP and IoT further. If you don't want to run your own infrastructure, Clayster is able to provide an XMPP Server and the discovery/provisioning platform for you. Feel free to reach out to rikard at clayster.com if you are interested to learn more about using XMPP for your next IoT project. www.clayster.com
I have changed the minimum requirement for TVPortal to MPExtended 0.6 (there is beta available). This enables certain useful features like delete. After tinkering with TVML quite a bit I ended up making the UI with UIkit directly. Though, it should look vary familiar to tvOS users. I’ve included some screenshots of completed screens below. I need to update the UI for one final screen and we are ready to release.
Summer vacations have been getting tougher in recent years. Airlines cut into your precious vacation time with their online check-in procedures and a dozen reminder messages, there is growing concern about airport security and Brexit has already put one large travel firm into liquidation leaving holidaymakers in limbo.
If that wasn't all bad enough, now there is a new threat: while you are relaxing in the sun, scammers fool your phone company into issuing a replacement SIM card or transferring your mobile number to a new provider and then proceed to use it to take over all your email, social media, Paypal and bank accounts. The same scam has been appearing around the globe, from Britain to Australia and everywhere in between. Many of these scams were predicted in my earlier blog SMS logins: an illusion of security (April 2014) but they are only starting to get publicity now as more aspects of our lives are at risk, scammers are ramping up their exploits and phone companies are floundering under the onslaught.
With the vast majority of Internet users struggling to keep their passwords out of the wrong hands, many organizations have started offering their customers the option of receiving two-factor authentication codes on their mobile phone during login. Rather than making people safer, this has simply given scammers an incentive to seize control of telephones, usually by tricking the phone company to issue a replacement SIM or port the number. It also provides a fresh incentive for criminals to steal phones while cybercriminals have been embedding code into many "free" apps to surreptitiously re-route the text messages and gather other data they need for an identity theft sting.
Sadly, telephone networks were never designed for secure transactions. Telecoms experts have made this clear numerous times. Some of the largest scams in the history of financial services exploited phone verification protocols as the weakest link in the chain, including a $150 million heist reminiscent of Ocean's 11.
For phone companies, SMS messaging came as a side-effect of digital communications for mobile handsets. It is less than one percent of their business. SMS authentication is less than one percent of that. Phone companies lose little or nothing when SMS messages are hijacked so there is little incentive for them to secure it. Nonetheless, like insects riding on an elephant, numerous companies have popped up with a business model that involves linking websites to the wholesale telephone network and dressing it up as a "security" solution. These companies are able to make eye-watering profits by "purchasing" text messages for $0.01 and selling them for $0.02 (one hundred percent gross profit), but they also have nothing to lose when SIM cards are hijacked and therefore minimal incentive to take any responsibility.
Companies like Google, Facebook and Twitter have thrown more fuel on the fire by encouraging and sometimes even demanding users provide mobile phone numbers to "prove they are human" or "protect" their accounts. Through these antics, these high profile companies have given a vast percentage of the population a false sense of confidence in codes delivered by mobile phone, yet the real motivation for these companies does not appear to be security at all: they have worked out that the mobile phone number is the holy grail in cross-referencing vast databases of users and customers from different sources for all sorts of creepy purposes. As most of their services don't involve any financial activity, they have little to lose if accounts are compromised and everything to gain by accurately gathering mobile phone numbers from as many users as possible.
Can you escape your mobile phone while on vacation?
Just how hard is it to get a replacement SIM card or transfer/port a user's phone number while they are on vacation? Many phone companies will accept instructions through a web form or a phone call. Scammers need little more than a user's full name, home address and date of birth: vast lists of these private details are circulating on the black market, sourced from social media, data breaches (99% of which are never detected or made public), marketing companies and even the web sites that encourage your friends to send you free online birthday cards.
Every time a company has asked me to use mobile phone authentication so far, I've opted out and I'll continue to do so. Even if somebody does hijack my phone account while I'm on vacation, the consequences for me are minimal as it will not give them access to any other account or service, can you and your family members say the same thing?What can be done?
Previous blogs on SMS messaging, security and two factor authentication, including my earlier blog SMS Logins: an illusion of security.
The 0.6.1 release of Libervia (Salut à Toi) is out, and it is a big one (despite of the minor version number).
The one which will follow — and is already well advanced — will be the « general public » release, or in other words we'll do what is necessary to have a release stable enough to be used seriously, and easy enough to install so the large public can enjoy it.
import plugin which can be extended, with "importers" for Dotclear and DokuWiki. Please note that, thanks to the use of standards, this can be useful even if you're not using Libervia to display your blog (it's compatible with Movim for instance)
in addition to the Dotclear import, its wiki syntax is available for SàT too, it comes with the already available Markdown and XHTML.
automatic subscription to PubSub feeds for new accounts, this avoids an empty pages for newcomers
MAM (XEP-0313) is implemented for PubSub, this allows to go back in publication history and to search in a more flexible and powerful way. Note that MAM is not available yet for instant messaging, it's planed for 0.7
better detection of repository revision
jp (CLI) has now an option --output with who you can specify the output format (e.g. JSON so parsing by an other command is easy).
jp: new command to import a blog (import), (re-)edit a new/old publication (edit) or have a preview in real time (preview). We have published a tutorial in french to explain how to publish on your XMPP blog with Vim, Emacs or whatever else, any help to translate to English welcome
jp: commands to manage your roster: get it (get), have statistic on it (stats), or clean it (purge)
jp: the command "message" has been moved to "message send" to be coherent with other commands, and because a "message get" should come in the future
Primitivus (console interface): paste detection (a validation is needed in this case, this avoid sending a bad text like a password by mistake), and handling of "bracketed paste mode"
Libervia (web interface): a new option "allow_registration" allows to deactivate the new account registration page, specially useful if you are alone on your instance, or if you create accounts by your own means
Libervia: a new pop-up with a counter is displayed if connection with server is lost
Libervia: new favicon with a notification counter
Libervia: connection with an external jid is fixed
Libervia: it is now possible to redirect pages or to integrate a local directory in Libervia. That's used to display directly the blog instead of the login page on www.goffi.org.
Libervia: TLS chains are now handled (hello Let's Encrypt)
Libervia (blog): tags are now handled, including search
we have also updated and improved our Docker images, so you can easily install and test Libervia. Please check the wiki page with the explanations.
This is a summary of 0.6.1, but it is not covering everything!
Salut à Toi is an ethical project is staying away from big proprietary companies, if you feel like helping it contact us by email (contact at salut-a-toi d.o.t org) or on our XMPP MUC room. We need more hands!
The 0.6.1 is the last one before the "general public" one and has its issues to be improved. Don't hesitate to report them so that the 0.7 will be rock solid :)
The 0.7 will go through beta testing phase, and will bring in particular:
Cagou, the new desktop/mobile devices frontend that we have promised after the success on our crowdfunding campain. The development is advanced, a post about it should arrive soon.
blog and messaging gateways
Beside all this, there is a good probability that we implement new end 2 end encryption system, in addition to the already available OTR. We specially think about OpenPGP and OMEMO.
Many people have now heard of the EFF-backed free certificate authority Let's Encrypt. Not only is it free of charge, it has also introduced a fully automated mechanism for certificate renewals, eliminating a tedious chore that has imposed upon busy sysadmins everywhere for many years.
These two benefits - elimination of cost and elimination of annual maintenance effort - imply that server operators can now deploy certificates for far more services than they would have previously.
For example, somebody hosting basic Drupal or Wordpress sites for family, friends and small community organizations can now offer them all full HTTPS encryption, WebRTC, SIP and XMPP without having to explain annual renewal fees or worry about losing time in their evenings and weekends renewing certificates manually.
Even people who were willing to pay for a single certificate for their main web site may have snubbed their nose at the expense and ongoing effort of having certificates for their SMTP mail server, IMAP server, VPN gateway, SIP proxy, XMPP server, WebSocket and TURN servers too. Now they can all have certificates.Early efforts at SIP were doomed without encryption
In the early days, SIP messages would be transported across the public Internet in UDP datagrams without any encryption. SIP itself wasn't originally designed for NAT and a variety of home routers were created with "NAT helper" algorithms that would detect and modify SIP packets to try and work through NAT. Sadly, in many cases these attempts to help actually clash with each other and lead to further instability. Conversely, many rogue ISPs could easily detect and punish VoIP users by blocking their calls or even cutting their DSL line. Operating SIP over TLS, usually on the HTTPS port (TCP port 443) has been an effective way to quash all of these different issues.
While the example of SIP is one of the most extreme, it helps demonstrate the benefits of making encryption universal to ensure stability and cut out the "man-in-the-middle", regardless of whether he is trying to help or hinder the end user.Is one certificate enough?
Modern SIP, XMPP and WebRTC require additional services, TURN servers and WebSocket servers. If they are all operated on port 443 then it is necessary to use different hostnames for each of them (e.g. turn.example.org and ws.example.org. Each different hostname requires a certificate. Let's Encrypt can provide those additional certificates too, without additional cost or effort.The future with Let's Encrypt
The initial version of the Let's Encrypt client, certbot, fully automates the workflow for people using popular web servers such as Apache and nginx. The manual or certonly modes can be used for other services but hopefully certbot will evolve to integrate with many other popular applications too.
Currently, Let's Encrypt's certbot tool issues certificates to servers running on TCP port 443 or 80. These are considered to be a privileged ports whereas any port over 1023, including the default ports used by applications such as SIP (5061), XMPP (5222, 5269) and TURN (5349), are not privileged ports. As long as certbot maintains this policy, it is generally necessary to either run a web server for the domain associated with each certificate or run the services themselves on port 443. There are other mechanisms for domain validation and various other clients supporting different subsets of them. Running the services themselves on port 443 turns out to be a good idea anyway as it ensures that RTC services can be reached through HTTP proxy servers who fail to let the HTTP CONNECT method access any other ports.
Many configuration tasks are already scripted during the installation of packages on a GNU/Linux distribution (such as Debian or Fedora) or when setting up services using cloud images (for example, in Docker or OpenStack). Due to the heavily standardized nature of Let's Encrypt and the widespread availability of the tools, many of these package installation scripts can be easily adapted to find or create Let's Encrypt certificates on the target system, ensuring every service is running with TLS protection from the minute it goes live.
If you have questions about Let's Encrypt for RTC or want to share your experiences, please come and discuss it on the Free-RTC mailing list.
There is increasing demand for SMS notifications about monitoring alerts, trading notifications, flight delays and other events. Various companies are offering SMS transmission services to meet this demand and many of them aggressively pushing their own proprietary interfaces to the SMS world rather than using the more open and widely supported SMPP.
There is good reason for this: if users write lots of of scripts to access the REST API of an SMS service, the users won't be able to change their service provider without having to change all their code. Well, that is good if you are an SMPP vendor but not if you are their customer. If an SMS gateway company goes out of business or has a system meltdown, the customers linked to their REST API will have a much bigger effort to migrate to a new provider than those using SMPP.
The HTTP REST APIs offered by many vendors hide some details of the SMS protocol and payload. At first glance, this may feel easier. In fact, this leads to unpredictable results when delivering messages to users in different countries and different character sets/languages. It is better to start with SMPP from the beginning and avoid discovering those pitfalls later. The SMS Router free/open source software project helps overcome the SMPP learning curve by using APIs you are already familiar with.
More troublesome for large organizations, some of the REST APIs offered by SMS gateways want to make callbacks to your own servers: this means your servers need public IP addresses accessible from the Internet. In some organizations that can take months to organize. SMPP works over one or more outgoing TCP connections initiated from your own server(s) and doesn't require any callback connections from the SMPP gateway.
SMS Router lets SMS users have the best of both worlds: the flexibility of linking to any provider who supports SMPP and the convenience of putting messages into the system using any of the transports supported by an Apache Camel component. Popular examples include camel-jms (JMS) for Java and J2EE users, STOMP for scripting languages, camel-mail (SMTP and IMAP) for email integration and camel-sip (SIP) or camel-xmpp (XMPP) for chat/instant messaging systems. If you really want to, you can also create your own in-house HTTP REST API too using camel-restlet for internal applications. In all these cases, SMS Router always uses standard SMPP to reach any gateway of your choice.Architecture overview
The SMPP support is based on the camel-smpp component which is in turn based on the jSMPP library, which comprehensively implements the SMPP protocol in Java. camel-smpp can be used with minimal configuration but for those who want to, many SMPP features can be tweaked on a per-message basis using various headers.
The SMPP gateway settings can be configured and changed at will using the sms-router.properties file. The process doesn't require any other files or databases at runtime.
The SMS Router is ready-to-run with one queue for sending messages and another queue for incoming messages. The routing logic can be customized by editing the RouteBuilder class to use other Camel components or any of Camel's wide range of functions for inspecting, modifying and routing messages. For example, you can extend it to failover to multiple SMPP gateways using Camel's load-balancer pattern.
SMS is not considered secure, the SMS Router developers and telecommunications industry experts discourage the use of this technology for two-factor authentication. Please see the longer disclaimer in the README file and my earlier blog about SMS logins: an illusion of security. The bottom line: if your application is important enough to need two-factor authentication, do it correctly using smart cards or tokens. There are many popular free software projects based on these full cryptographic solutions, for example, the oath-toolkit and dynalogin.
We are proud to present the latest step on our road to the MongooseIM 2.0.0 release. Enjoy MongooseIM 2.0.0beta1 - now available on GitHub, and see the scope of changes that made us jump to 2.0.0.What's new
MongooseIM 2.0 is about new features. We added support for several XEPs and we created our own open extensions to the XMPP protocol developed as solutions to customer problems.New XMPP extensions Publish Subscribe for real-time non-chat applications
Over the last year or more, many customers required the XEP-0060: Publish Subscribe extension. We decided to restore the support for it along with XEP-0163: Personal Eventing Protocol. This opens doors for many use cases which were hard to achieve before. It is now possible to seamlessly integrate MongooseIM with services such as Movim or other micro blogging engines based on XMPP. More generally, it opens a wide range of possibilities in the real-time application space.Blocking Command for enhanced and simpler privacy enforcement
In many cases XEP-0191: Blocking Command (which is simpler version of XEP-0016: Privacy Lists) does the required job which is blocking a communication with specified users. These two functionalities do not exclude each other, so it is still possible to have more sophisticated privacy settings when required. The added support for this extension makes the life of a client side developer significantly easier.Client State Indications for bandwidth and battery savings
The XEP-0352: Client State Indication is a very important extension for mobile clients. It allows the application to notify the server that from now on it is in an inactive state. The server then reduces the amount of data sent to the user which lowers the battery and bandwidth consumption.Innovative extensions MUC light for contemporary and simpler group chats
For over 2 years now we noticed that mobile-first and mobile-only approaches in application development are the most popular. For the majority of our customers the current MUC as described in XEP-0045: Multi-User Chat was too big and too focused on presence, so they had to invent workarounds.
This is why we came up with a simpler, presence-less version of MUC: "MUC light". When the user goes offline, the user automatically leaves the MUC room, but stays subscribed to the MUC light room. When back online, the user must automatically re-join the MUC room in order to receive the latest messages (potentially losing some of them), while, using the MUC light room feature, it is a matter of only retrieving a message archive (no message loss).
We believe this is a good step towards making a group chat more straightforward and up-to-date from both client developer’s and user’s perspective. There is already some traction and support for the MUC light.Token-based authentication
Mobile clients (and some laptops with wifi) can lose the connection more often than desktop clients. In such cases it is required to quickly reconnect (including re-authentication) the client when Internet connectivity is back. One way of doing this is to store user's password on the device and use it while reconnecting. This is not the most secure solution. In order to avoid the password storage on the device we propose a new authentication mechanism based on tokens obtained from the server. More info is available hereOther changes and improvements Parallelised tests
As a team we always think about how to test what we create and how to improve existing tests. Adding new test cases may have its downfalls, especially when you want feedback quickly or you run on a public infra like travis-ci.org and you are closing the max allowed job time (which is around 50 min). In order to optimise we decided to rewrite many of our test suites to run test cases in parallel. This shortens a single job execution significantly but there is still room for further optimisations which we're going to do.Many more valuable improvements
There is also many other smaller and bigger improvements. All of them can be found in the changelog.Mobile libraries with feature parity to faster time-to-market
Over the last couple of months, we have significantly contributed to third party, opensource XMPP libraries. We are targeting Android and iOS, respectively with Smack and XMPPframework. Basically, we are offering feature parity with MongooseIM. This enables mobile apps developer and sysadmin to develop and integrate MongooseIM features much faster and more easily in their apps.Movim, real-time social media over PubSub
We have had a lot of fun playing with Movim on top of MongooseIM, in order to test a decentralised social media network, working over PubSub. We have contributed a Docker image to the project.Thanks
We will release a 2.0.0beta2 in the coming weeks, adding even more value to MongooseIM, enabling you to significantly lower costs and time-to-market. Stay tuned.
Meanwhile, get our fresh news: subscribe our announcement mailing list.
Ignoring the UI, if you are interested in seeing where HDTV streaming on Apple TV via Media Portal is. I have made a small video. I hope to have live TV streaming working next followed by ad skipping and seeking (with comskip data) .