Catalyst IT Limited  
Planet Catalyst
 

22 September 2016

Catalyst Blog

face

Burning Samba with perf and FlameGraph

Samba implements many things, but in this case we look at Samba as an Active Directory DC.

In that role, Samba now has a quite credible implementation of the required protcools, and so is deployed in Education, Governements, large corporations and small buisnesses around the world.

One of the great things about our Samba users is the scale they are willing to push Samba to.  With Samba now well established as a reliable solution, our enthuisatics users are now deploying it at larger and larger scales, and rightly expect that it continues to 'just work'.

It is in this environment that our team here at Catalyst has been working to keep Samba performing to those expecations, and this blog will look at what we have achived for the Samba 4.5 release.

It all started with two issues raised by our clients, which turned out to dove-tail really well:
 - Samba replciation performance with a large number of linked attributes.
 - Samba runtime add/remove performance with linked attributes.

This blog will look at how we addressed the first issue, in replication.

We first worked to reproduce the situation found by the client, that is a domain with a large number of users and in particular users in groups.  Due particularly to the second issue, this was quite slow, so we pre-created such a database for re-use.

Then the replication operation was run under 'perf record'.  The linux perf/perf_events tools from the Main Page are a low-overhead sample-based profiling system, allowing us to explore where (statistically) our program time is being spent.

Replicating the problem statement from the client, our domain has a larger number of group members, and these memberships are described as
linked attributes in AD.  Under perf record, we ran 'samba-tool drs clone-dc-database' to emulate replication without making any
modifications to the domain.

To determine where the time is being spent, the output was then visualised with the FlameGraph tool by Brendan Gregg.


As you can see in this image above, it turns out that Samba's handling of these is quite suboptimal in two ways:

 - A number of the processing steps are clearly being done in a tight loop (as GUID_compare() and ldb_val_equal_exact() are very fast routines when run only once.
 
 - The processing is done in ldb_transaction_commit, so this means it is with the database transaction lock held.  In this case no other process can make writes to the database while this processing is being undertaken.

One of the most helpful things about these flame graphs is they are dynamic: they include JavaScript, and we and you can drill down into them, and use them as a map to investigate the source code.  We can follow from replmd_prepare_commit() down to ltdb_modify_internal(), and see the real problem:

Helpfully, described in the comment as O(n^2), we need to avoid doing this processing.  Thankfully, in this case the solution was not a change to a better algorithm such as a merge sort, but to [
avoid the check entirely!

The difference this has made is quite clear as seen above, as an entire toung of flame has just vanished.  This graph also differs in that we changed the trace from being kernel-wide to just the client process, once it became clear that the CPU time is only problematic in the client.

We attacked the GUID_compare() stack next, with a simple change to apply the linked attributes earlier during the replication, so as to avoid building and so traversing a long list. Again, the results are quite dramatic.

However, there is still clearly a problem: get_parsed_dns() is being called way too often, and it spends almost as much time cleaning up the memory allocated as doing actual work.

While this is certainly worthy of attention, we left this particular line of attack, and instead focussed on why the underlying routines are so expensive.  This matters because while clearly over-used, these routines are also used incredibly often in the rest of the codebase. We focussed on why the NDR parsing of GUIDs and SIDs.  Both carried an incredibly high load for memory allocation compared to the work done, and so we added routines that perform NDR parsing without memory alloation.

We also examined why talloc calls, and particularly talloc_free() calls are so expensive, and talloc 2.1.8 contains these improvements.

After all this work, for this particular example, we can now do replication in around half the time previously taken!

19 September 2016

Catalyst Blog

face

Recurring Payments with Drupal Commerce and PayPal

by Andrew Boag

During the build of one our Catalyst as-a-service offerings – CourseBank – we implemented a recurring Credit Card billing solution, allowing customers to fully provision a premium account.

Catalyst has been involved in a number of Credit Card and mobile payment solutions over the last 15 years. And things have certainly changed hugely in terms of the best practices and the number of payment gateways on the market.

It has always been considered a dangerous practice to retain a copy of the client's Credit Card details in your own application database. As it makes you potentially liable for the wholesale theft of Credit Card details – something that may likely hit the news and cause havoc. Saying that, in the past it was not unheard off to implement recurring payments by simply storing the Credit Card details inside your application and firing off an API call with a payment request every month. Not something that Catalyst recommends.

These days, the initiatives in the Payment Card Industry Data Security Standard (PCI DSS) standard since 2004 mean that there are concrete rules that systems need to comply with around the protection of card holder data. For the most part, these initiatives are for the better of both retailer and customers, reducing the overall risk of Credit Card theft and fraudulent transactions.

The safest and most usual model when integrating Credit Card payments into your online store is to not “see” or process the Credit Card details at all, instead integrating with an external payment gateway that processes the entire transaction in a work flow something like the following:

  • At the point that customer has already selected their product in your web store they click on “Proceed to Checkout” link. Taking them to external payment gateway.
  • At this point they have left the web store application. And are now located at a payment gateway (e.g. PayPal) – where customer sees the items they are purchasing again, with a total cost.
  • Now customer enters their Credit Card details and confirms the purchase. This may include some identity validation via their bank such as an SMS being sent to card holder with a code that they must enter.
  • Once customer has completed payment, they are forwarded back to our online store with a “Payment Successful” message.
  • Customer will likely also receive some payment notifications by email after the transaction.

Catalyst's Recurring Payment Requirements

Our CourseBank product is a traditional online service offering, with customers providing their Credit Card details only once during sign up. Then they are billed every month until they cancel the service.

Catalyst chose PayPal as our payment gateway partner.

Why PayPal?

Catalyst made this choice based on investigations, experimentation and experience. The PayPal service offering gave us a suitable feature set to implement recurring payments, and to integrate this with our Drupal 7 application. We use Drupal as the gateway for subscription, payment workflow and ongoing account management.

Conveniently, it is PayPal that schedules and handles the recurring payments, notifying us after each payment success or failure via a PayPal IPN message. Drupal does not trigger any payment via a monthly cron job or script. Catalyst have had negative experiences with

If you are considering PayPal, you should investigate if you will need to set up a PayPal Business account in order to accept payments for your business. These are also sometimes referred to as Merchant Accounts. You will also need to make sure that you have everything in place to clear funds from your PayPal account into your bank account and to adhere to all taxation requirements. Talk to your accountant or finance team and make sure you test things thoroughly (see section on Testing below). The rules for this will vary from country to country and are beyond the scope of this article.

PayPal's Instant Payment Notification (IPN) system is a very solid feature and one of the drawcards of . IPN's facilitate “reporting home” to our Drupal application about scheduled payments. It is very well designed and mature and we liked it for the following reasons:

  • IPN notification web service receipt was already implemented in Drupal Commerce module. Meaning it was easy to implement application specific hooks based on events e.g. restricting access to premiume when a failed payment IPN was received.
  • IPN notifications include all of the required success and failure payment events.
  • IPN notifications have a robust retry architecture in the event that our application is off line or unavailable for any period of time i.e. notifications from PayPal are not lost or discarded.
  • PayPal web interface allows us to easily investigate payment events in the case that there was any question around any payment event. We have all the information at our fingertips.

Drupal – Commerce PayPal

Great thing about Drupal is the number of contrib modules already published for our use. Meaning that we, as Drupal developers, can concentrate our efforts on building the innovative and bespoke functionality our project needs, and not re-invent the integration wheel.

The Drupal Commerce module already has integration with PayPal for payment processing, and already includes an IPN listener, to catch asynchronous payment messages.

QA and Testing – This Time For Cash

We all understand the value of incorporating proper Quality Assurance (QA) and testing into the build cycle of any enterprise application. This is the best way to reduce painful cleanup and issue resolution of functional bugs making their way to production.

But where automated payments are involved, the potential risks and implications around implementation bugs are considerably more far reaching.

As well as all the standard functional testing and unit testing, Catalyst have some extra best practices that we have included when we are working with Payment System integrations:

  • Be aware that test payment gateways provided by providers (including PayPal) are not enough to test the end-to-end behavior of all functionality. Of course developers and QA should make use of them. But part of the application testing needs to involve real payments in sandboxed environments. This is a little non-intuitive.
  • We advise getting some pre-paid credit or debit cards for testing purposes. To trigger real transactions as part of the QA phase to trigger expected events such as insufficient funds and cancellation of transaction after the event (via phone call).
  • Involve the whole dev and infrastructure team in discussions to make sure that there is no chance that non-production environments can ever trigger real payments.

Hope this information is of use, of course if you have any further questions please reach out.

NOTE: Please do not consider this article as an endorsement for PayPal above all other payment gatewards. Catalyst has no commercial commitment to PayPal other than as a standard customer. There are a number of considerations around the selection of Credit Card payment facilities that are outside the scope of this blog.

06 September 2016

Catalyst Blog

face

7 Themes of Catalyst

by Don Christie

OS//OS Talk – 7 Themes of Catalyst

I gave a talk at the second Open Source Open Society conference in Wellington recently. I was asked to talk about how Catalyst had grown and evolved over the last 20 years and was allocated seven minutes for my talk. For a 20 year old company that's quite a lot to cover in a short time, especially as it is my favourite topic.

So, I decided to do this by touching on seven themes that describe our company and give an essence of who we are and what drives us. This seemed to resonate with the audience and the general thrust of the conference, so I am grateful to have had the opportunity to present.

Here are my preparation notes:

1. Catalyst

Catalyst in 2016 is in its 20th year of operation. We are a company that makes free and open source software (free software from now) work beautifully for our clients. People are good enough to pay us for this work and to develop even more free software. Catalyst employees have contributed to over 170 projects, often as a result of work carried out for our clients, with their permission.

With 250 staff and with offices in Wellington, Christchurch, Auckland, Sydney, Melbourne and Brighton (UK) we are the largest free software company in Australasia.

Recently we have built a public cloud in New Zealand, entirely based on free software and set up to compete with the likes of AWS and Azure. We will be opening our third region late in 2016. This is an achievement we are all very proud of and builds on capabilities across the company.

2. Purpose before profit

All organisations need a purpose. There really isn't much point in existing just to print money. We have a central bank to do that. It's a clear common purpose that gets us out of bed in the morning.

The free software ethos and mindset is in our bones. Our company promotes the free software mindset and way of operating in the widest possible sense.

We believe these collective values drive how we use technology and can lead to a reduction in imbalances in society.

Profit is important. Not least, because it demonstrates that our approach is a legitimate way of doing business and not a fringe activity.

3. The power of technology is asymmetric, free software can fix this

We talk a lot about the growing divide between rich and poor. But what we don't talk enough about is the gaping chasm in power between suppliers of technology platforms and consumers. In the 21st century it is impossible to participate in our communities without using technology. However, many providers of technology force us to participate in a way that most people would rather not.

They control our private information, they dictate what we are allowed to see and do and force us to adopt patterns of behaviour the aggregate flows of revenue to fewer and fewer corporations.

Not just that, but the pace of change is so fast that the technically less literate, often older people, are not able to keep up and therefore they are more and more excluded from the world that many of us take for granted.

Governments have near unfettered access to our personal lives in a way that totalitarian regimes of the past could only dream of.

Free software (especially strong copyleft licensing regimes) attempts to address this balance by placing the right to control technology and how it is used firmly back in the hands of users. The role for technologists in this regime is to help and serve our communities so that they are able to benefit from this right.

4. Sharing is a good thing

It's just like your mother used to say, sharing is good. Around the time we leave primary school we start getting brainwashed into believing the opposite. Sharing becomes “cheating” and later, stealing, a heinous criminal activity.

Software “End user licence agreements” - EULAs can literally take days to read but mostly they cover the different ways sharing is wrong and punishable.

But our society doesn't function like that. We need to share. Not only because we all “stand on the shoulders of giants” but because every day we require support and knowledge from those around us and more broadly, around the world.

Free software, creative commons and open data all recognise the importance of being able to share.

5. Permissionless innovation

Catalyst has coined the by-line “freedom to innovate” to express this concept.

Free software has created this extraordinary opportunity for innovation to take place, building on the works of others without requiring anyone's permission to do so.

To use and further develop free software you don't need:

  • Contracts
  • Lawyers
  • Heads of agreement
  • Memorandums of understanding.

Koha, the GPL licensed library management system was developed in 1999 for Horowhenua Library Trust in Levin, New Zealand. The code was released for anyone to use. The Koha development team received their first patch from a large car manufacturer in Detroit who had decided to use the software to catalogue their thousands of manuals. The took the code from the Internet repository, changed it, ran it and contributed some changes back to the core product. All without third party involvement. Now, of course, Koha is used in thousands of libraries across the globe, translated into dozens of languages and is the most feature rich Library Management System in the world. The great part, it is still free.

6. Don't be passive

Positive change is not something that happens by accident. We have to make choices and sometimes those choices are not convenient. They require us to learn or to go an extra step. But, in Catalyst's experience,making harder choices always pays off.

Adopt these changes for yourself:

Switch to Firefox as your default browser on all devices. Firefox is the only mainstream free and open source browser. It keeps the others honest. It's very good.

If you are a developer, develop for Firefox first. Your websites will be more standards compliant and more likely to work across all other browsers. If you use IE or Chrome first you are likely to run into compatibility problems.

Try Ubuntu or Linux Mint on your desktop. These are beautiful fully featured and far less confusing to users as the proprietary equivalents.

Use LibreOffice rather than MicrosoftOffice. It uses open document formats by defaults as a bonus.

If you are releasing free and open source software, chose a licence that respects and perpetuates the rights of users. The copyleft, GPL, licence family is a great start.

If you are not prepared to do any of the above...consider this, why are you here?

7. Freedom always wins in the end

It is a condition of human kind. History tells us that no matter how dark the days become we humans are always struggling for freedom.

In my shortish lifetime alone I have seen the rolling back of Communist inspired totalitarianism in Eastern Europe, the end of Apartheid in South Africa and the legalisation and legitimisation of homosexual relationships. Despite occasional setbacks, democracy is the default state for most of the world's countries. This is an issue that transcends normal party politics. As far as I can tell, all political parties have their authoritarian tendencies and most have more freedom loving wings. The latter groups need to be encouraged and supported.

Technology that has freedom baked in also wins. People forget that there were proprietary, non-free versions of the Internet – such as Compuserve. But the Internet, that was based on ideals and protocols and software that was open and free won, and keeps winning. Linux is now used on more devices than any other operating system. Free and open source software drives most new startups as well as the biggest technology companies in the world.

People like to be on the winning team and when history comes to judge you, be remembered for being on the side of freedom.

27 July 2016

Kristina Hoeppner

face

A very secure MUGSE meeting

MUGSE logoOn Monday, 25 July 2016, the latest MUGSE (Mahara User Group South England) meeting took place at the Defence Academy on its Shrivenham campus. I was fortunate to have been able to attend because I think it will be the most secure Mahara user group meeting ever. But of course, the attendees were also a big draw. 🙂

So what about the security?

To get it out of the way because I’m sure you are wondering what’s it like to have a meeting on a military campus, let’s talk about it first.

Sam Taylor and Aurélie Owens from the Flexible Learning Support Centre at Cranfield Defence and Security hosted the meeting and did a great job explaining all the necessary details. We had to sign in at reception and received a badge that allowed us to wander around the campus without an escort (not that we attempted because it’s such a huge campus and you could wander into a minefield or in the line of fire if you didn’t read all the signs). We were not allowed to take pictures outside of our meeting room, and also had to wear appropriate attire as we were having lunch at the Officer’s Mess Hall.

It was interesting to see people dressed in uniform everywhere, though there are also staff like Sam and Aurélie who do not wear uniform.

We were lucky not to have any sirens go off during our time on campus or hear any military drills and practices as it was very quiet on campus with students having finished their theses and awaiting the assessment outcomes.

So you said MUGSE?

25 keen Mahara users from across the south of England all the way from Cambridge in the east to Southampton on the coast and Nottingham in the north (not quite south England anymore, but she will be taking over from Sam while she is going on maternity leave later in the year) braved the summer weather and in one case even came back from vacation to attend this summer MUGSE meeting. It was a great diverse group of people primarily from the higher and further education sector. Some were newbies and others experienced users.

So what did you do?

We began the day with a short workshop to get everyone talking and brainstorming. We had 5 topics of which 4 were discussed in groups allowing everyone to put their heads together and see if they could come up with a solution or an idea for a  change in Mahara. The topics discussed were:

  • How can you integrate assessment?
  • How can you improve usability?
  • How can you integrate Microsoft Office 365?
  • How can you bulk update your group membership (not discussed in wider group)?
  • What are the benefits of using Mahara?

We then heard short case study reports from various institutions of higher education, I presented the state of SmartEvidence so far, we looked at some easy ways to improve usability in Mahara locally and then had a Q&A session with which we ended the day.

Workshop

Assessment

Assessment is becoming an increasingly important topic for Mahara users as they wish to tie in competency frameworks with the portfolios. We looked at the first prototype of SmartEvidence later in the day. The group shared their experiences so far and came up with questions to be discussed further.

MUGSE group discussion

MUGSE group discussion

Usability

The group raised a number of questions that were in good part related to the other groups. Some of the questions were also tackled in the form of case studies by the presenters that came after the workshop who provided good insight into how Mahara is used around southern England currently.

Usability questions raised at the MUGSE July 2016 meeting

Usability questions raised at the MUGSE July 2016 meeting

It was noted by the group that professional development plays an increasingly important role for the successful implementation of ePortfolios.

Office 365 integration

Two participants discussed the Office 365 integration and also general questions around the integration of other external resources. Integrating with external repositories is beneficial for those that only have limited storage capacity. However, what happens when the students don’t have access to these repositories anymore? Also campuses such as the Defence Academy are in a unique security position in that they are not allowed to use external storage and everything needs to be put on its on-site servers that are not the cheapest around.

Update: If you want to take an early look at the integration between Mahara and Office 365, please contact Gregor Anželj. He has worked on the integration and can make the plugin available for testing before its final release.

Bulk update group membership

While this topic was not discussed in a bigger group, Nick and I did brainstorm possibilities. I mentioned that Mahara 16.10 will have the possibility to update any group’s membership via CSV file and not just the membership of the groups that were created by CSV as is currently the case.

However, it would be nice to have some bulk options available where users can quickly update their own group membership status. Similarly, it would be good to have some bulk options also available for portfolio pages.

Benefits of using Mahara

The members of this group discussed benefits and noted importantly that competent support is required for uptake at an institution and successful implementation. The group pointed to the excellent support that Cranfield University receives from Sam and Aurélie for example, who provide valuable advice to lecturers from the start of a project to the finish when brought in to support the lecturer.

Case studies

Cranfield University

Sam and Aurélie presented an update to their presentation at MootIEUK in March. They had now run a second course in which their students worked on problems collaboratively and used a group portfolio to report their findings and reflections. They had learned a number of things from the first pilot and made changes accordingly to their course design as well as how they worked with Mahara, which were successful and proved beneficial to their participants.

A notable difference in Mahara was the use of collapsible blocks for the portfolio content. As the students needed to submit their portfolios to Turnitin, everything had to be on one page. Turnitin does not handle zip files well, but it can read the printed version of the portfolio reasonably OK.

Of course putting an entire portfolio consisting of multiple sections with quite a bit of text on one page results in endless scrolling. That’s where the collapsible blocks come in: It’s easy to jump between the sections, but when it does need to be printed, all sections can be opened once and a print copy can be made.

They did stress to the participants though that the assessors would look at the nicely formatted portfolio in Mahara as well and not only the plain one in Turnitin. Layout and overall organisation still counted, and staff found the portfolios very clear and well structured.

Next steps at Cranfield University

Next steps at Cranfield University

Southampton Solent University

Roger Emery from Southampton Solent University reported on the use of templates for health care students and how his team improved the support for them when they are not on the same campus. In the past, students didn’t always know whom to contact for support, but now they have a structure in place that is clear and the learning technology team is more efficiently used.

There are peers, who already had to do their portfolio, available to help students. The learning technology team does check in with them to ensure that they aren’t giving incorrect advice. The next level of support is the assessors, and Solent University found that it helps when they create their own portfolio to see how their tasks work and what is doable and how it is doable by the students. And lastly there is the tutor who can be consulted. Peers, assessors and tutors then have the direct line to the learning technologies team in case they exhausted their other resources.

The use of peers as initial support for students (and instructors) works well for a number of institutions. It is good to see students supporting students and gaining more knowledge at the same time.

While the team does use templates, they found that this can be dangerous at times as lecturers then don’t explore the system themselves and learn how to use it.

Roger pointed to the help pages of Solent University where there are also some portfolio examples available.

Cranfield University

Sam showed us some of the support resources that she’s put together for her students in Moodle and how they link back to Mahara. While they use existing resources, they also create some resources themselves to focus on the specific tasks learners need to do and make it easy for them to find that information.

The day before, Sam was showing me her support resources and we looked into deep linking to Mahara pages from within Moodle so that she wouldn’t have to provide a secret URL to the support pages anymore, but could link to them and log the students into Mahara at the same time. That is helpful because then students don’t necessarily have to return to Moodle and click on the special login link, but are logged into their portfolios automatically.

University of Brighton

Nick Feather from the University of Brighton gave an update on the ePortfolio work that is happening at his university. Like Solent University, Brighton uses templates in health and humanities. They use tables extensively to provide an overview of what part of the portfolio has already been filled in and where there is still evidence missing.

Sparsholt College Hampshire

Ursula Bailey from Sparsholt College uses portfolios with zookeepers in training. They used to work with paper-based portfolios, but sending them across the UK to the various assessors proved very expensive. That’s why they looked into going digital. They started out using templates, but don’t anymore because they were too close to the paper-based versions and didn’t give the students enough freedom. For Sparsholt College it was crucial to get the external verifiers on board.

There are still a few things to improve and simplify. For example, right now students still need to collect the signatures of their mentors and tutors and put them into their portfolios to indicate that they’ve passed a unit. However, maybe in the future it would be enough for the tutors and mentors to provide feedback to indicate that they’ve reviewed the student’s work.

Sparsholt College also uses a manually created overview page showing where evidence has already been collected and whether it’s been verified or not.

Kings College, London

Charles Kasule from Kings College presented on the varied use of Mahara in 4 different contexts:

  • King’s Learning Institute
  • Florence Nightingale Faculty of Nursing and Midwifery
  • Modern Language Centre
  • King’s Experience
Charles Kasule presents on KCL's use of Mahara

Charles Kasule presents on KCL’s use of Mahara

The reflective element is very important in the King’s portfolios. One example that we saw was a lecturer’s reflection on preparing for one of his modules. He showcased his progress and the various stages.

The College set up a portfolio working group in which learning technologists and academics get together to discuss the implementation of portfolios. The involvement and support from the academics from the start is very important for the success of the implementation.

Catalyst

Initially, I was going to showcase some examples of using templates, but since the majority of the previous presenters already showed how well templates can or cannot work, we decided to switch gears on the spot and briefly demo SmartEvidence instead. A number of presenters had tables in their portfolio templates that students needed to fill in manually. Now that’s where SmartEvidence can come in very nicely I would say.

The first patches made it into code review, and I’ve been testing them over the last couple of weeks to check out the initial technical implementation. There are a few more things to finish before we can commit everything to Mahara, but I’m already very excited about the current state of progress. I think I’ll create a short video soon to show where we are at right now.

It was good to do this first demo and present the current functionality to a group of users and see how many of them are very excited about what is coming up.

Usability

Improving the usability of Mahara is an ongoing topic and as the discussion by one of the groups showed at this MUGSE meeting, a very important one that needs to be tackled from a number of perspectives.

Roger shared how Solent University is making improvements on its environment locally without much coding by primarily changing language strings. He showed how they have changed the help files in Mahara to make them more suitable for their students providing information on what the functionality is, how it can be used and seeing an example.

Not everyone is aware that any language string in Mahara can be changed with the help of a local language file. Thus, adapting the language to the one that is used at an institution is fairly simple and does not require any coding background. The workshop material that Roger had put together for last year’s Mahara Hui helps get others started with that.

Mahara clinic

At the end of the day we had a Mahara clinic where any questions could be asked. The one big question I remember was around the mobile app and whether we’ve made any progress on that. So I could give an update on Mahara Mobile. Hopefully, we’ll have an alpha version available by the end of the Northern Hemispheric summer. A few bugs are eradicated and the theme is being worked on.

The app is still – like its current predecessors – going to be based on the premise that we aren’t replicating all of Mahara in an app, but see the app more as an extension to the website and make it possible for learners to record their learning evidence offline. The new app will allow learners to connect it to any Mahara site no matter the authentication method (tested will be built-in authentication methods). That’s a big improvement as currently, the apps can only be used with internal Mahara authentication. With more and more institutions setting up LDAP or single sign-on, that’s not a viable option.

I’m excited about the new app and will announce the test version as soon as we have it available.

So was that all?

The day at Cranfield University was jam-packed with conversations and presentations around the use of Mahara at a number of institutions and provided new as well as experienced users a good overview of what is going on in South England. There is a lot of good portfolio work being done that supports students and instructors in their learning and in a number of cases also supports collaboration and makes learning enjoyable. One lecturer who had observed group work noted that the students had never laughed so much during an activity before.

The local Mahara user group meetings give people who may not be so active in the online community the chance to get together relatively easily (some did travel more than 6 hours in total though to be able to attend) and exchange ideas as well as get to know each other a bit better. Being able to bond and network face to face is still very important. I am looking forward to seeing how the individual projects develop and change over time.

Check out Twitter for the tweets from the event as well as future updates.

24 July 2016

Andrew Ruthven

Allow forwarding from VoiceMail to cellphones

Something I've been wanting to do with our Asterisk PBX at Catalyst for a while is to allow having callers that hit VoiceMail to be forwarded the callee's cellphone if allowed. As part of an Asterisk migration we're currently carrying out I finally decided to investigate what is involved. One of the nice things about the VoiceMail application in Asterisk is that callers can hit 0 for the operator, or * for some other purpose. I decided to use * for this purpose.

I'm going to assume a working knowledge of Asterisk dial plans, and I'm not going to try and explain how it works. Sorry.

When a caller hits * the VoiceMail application exits and looks for a rule that matches a. Now, the simple approach looks like this within our macro for handling standard extensions:

[macro-stdexten]
...
exten => a,1,Goto(pstn,027xxx,1)
...

(Where I have a context called pstn for placing calls out to the PSTN).

This'll work, but anyone who hits * will be forwarded to my cellphone. Not what I want. Instead we need to get the dialled extension into a place where we can perform extension matching on it. So instead we'll have this (the extension is passed into macro-stdexten as the first variable - ARG1):

[macro-stdexten]
...
exten => a,1,Goto(vmfwd,${ARG1},1)
...

Then we can create a new context called vmfwd with extension matching (my extension is 7231):

[vmfwd]
exten => 7231,1,Goto(pstn,027xxx,1)

I actually have a bit more in there to do some logging and set the caller ID to something our SIP provider will accept, but you get the gist of it. All I need to do is to arrange for a rule per extension that is allowed to have their VoiceMail callers be forwarded to voicemail. Fortunately I have that part automated.

The only catch is for extensions that aren't allowed to be forwarded to a cellphone. If someone calling their VoiceMail hits * their call will be hung up and I get nasty log messages about no rule for them. How do we handle them? Well, we send them back to VoiceMail. In the vmfwd context we add a rule like this:

exten => _XXXX,1,VoiceMail(${EXTEN}@sip,${voicemail_option})
  same => n,Hangup

So any extension that isn't otherwise matched hits this rule. We use ${voicemail_option} so that we can use the same mode as was used previously.

Easy! Naturally this approach won't work for other people trying to do this, but given I couldn't find write ups on how to do this, I thought it be might be useful to others.

Here's my macro-stdexten and vmfwd in full:

[macro-stdexten]
exten => s,1,Progress()
exten => s,n,Dial(${ARG2},20)
exten => s,n,Goto(s-${DIALSTATUS},1)
exten => s-NOANSWER,1,Answer
exten => s-NOANSWER,n,Wait(1)
exten => s-NOANSWER,n,Set(voicemail_option=u)
exten => s-NOANSWER,n,Voicemail(${ARG1}@sip,u)
exten => s-NOANSWER,n,Hangup
exten => s-BUSY,1,Answer
exten => s-BUSY,n,Wait(1)
exten => s-BUSY,n,Set(voicemail_option=b)
exten => s-BUSY,n,Voicemail(${ARG1}@sip,b)
exten => s-BUSY,n,Hangup
exten => _s-.,1,Goto(s-NOANSWER,1)
exten => a,1,Goto(vmfwd,${ARG1},1)
exten => o,1,Macro(operator)

[vmfwd]

exten => _XXXX,1,VoiceMail(${EXTEN}@sip,${voicemail_option})
  same => n,Hangup

#include extensions-vmfwd-auto.conf

And I then build extensions-vmfwd-auto.conf from a script that is used to generate configuration files for defining accounts, other dial plan rule entries and phone provisioning files.

With thanks to John Kiniston for the suggestion about the wildcard entry in vmfwd.

18 July 2016

Kristina Hoeppner

face

The Vienna connection: Linking Mahara and edubreak

On 11 and 12 July 2016 I had the chance to finally meet a few Mahara community members with whom I had been in touch for numerous years at a meeting of the European PREPARE project. It’s been really nice to not only communicate via email, IRC or webinars, but in person.

Gregor Anželj, the community’s most prolific plugin developer, came from Slovenia, Klaus Himpsl-Gutermann from PH Wien, who had written his PhD thesis on the integration of Moodle, Mahara and MediaWiki,  hosted the meeting, and Andrea Ghoneim from Danube University Krems, who has been participating in a number of EU funded ePortfolio projects, joined us on the 11th. But that was not all. The following also attended (some of) the sessions (thanks Reinhard for providing all the links on your blog post): Erich Schönbächler (Teacher at Primarschule Feusisberg, Switzerland), Günter Nimmerfall (Responsible of the Service for Media and Technology, PH Tirol), Herbert Muska (ZID, PH Wien), Johannes Metscher (Ghostthinker, Augsburg, Germany), Klaus Hammermüller (Verein Offenes Lernen [Registered society “Open Learning”, Wien), Peter Groißböck (PH Niederösterreich), Reinhard Bauer (ZLI, PH Wien), and Wolfgang Greller (Vice Rector for Research and Quality Assurance, PH Wien).

Settling in for the first day at PH Wien

Settling in for the first day at PH Wien. Pictured are (left to right): Herbert, Reinhard, Klaus, Johannes, and Peter.

Day 1

We started on the 11th with a series of short presentations to bring everyone up to speed on the use of Mahara in the ATS2020 project, at PH Wien and PH Niederösterreich. I gave a short overview of the bigger projects that we are planning for Mahara in the coming 2 years, and we saw a brief introduction of edubreak.

ATS2020

Andrea discussed some aspects of the ATS2020 project (some information in German and English) and mentioned that teachers, who were introduced to working with portfolios, appreciated having sample lesson plans available to give them an idea of how to proceed and what to cover. Students needed guidance in giving feedback and also for evaluating themselves. When the students assessed themselves they were often stricter than their teachers.

While the project focuses on the assessment of transversal skills, it does look into the lifelong aspects of portfolios. Thus, they see that there is a problem when systems are connected to the portfolio as students may keep the access to their portfolio, but won’t always retain access to the other systems that were connected during their schooling. There is yet to be found a good solution to overcome this issue.

Andrea presents aspects of the ATS2020 project

Andrea presents aspects of the ATS2020 project

Integrating systems is very useful to allow for improved work flows and avoid duplicating of data, especially when it is to be updated. However, there is no solution to download / keep all off-site data when a student wishes to do so. A “disconnect and keep all data” button is missing.

PH Niederösterreich

Peter reported on the use of Mahara at his teacher training college. It is called “Sammelmappe” for which the English translation of “binder” doesn’t quite cover it. It’s essentially a place where you put everything that you wish to keep. Having connected it with LDAP, students have hardly any support queries. Keeping the number of passwords low is a definite plus. In the practical teacher education (“Schulpraxis”) it’s become the default to have an electronic portfolio, and students complete a reflection and development portfolio. The trend is also to only use Mahara. There, students find all necessary forms and information in regard to their study program.

The same is true for the Master’s that Peter is involved in. Having everything on a single platform instead of distributed over Moodle and Mahara, is beneficial for the students who are not always very tech savvy. Over the years, they adapted their portfolio practices. In the beginning, there were a lot of reflections. This became a huge task for faculty to provide feedback as they often have many students. Thus, they dialed back on the number of reflections and now have them only for milestones.

Peter walked us through one of his portfolio pages that he had set up for the semester tasks. He mentioned that he uses blocks that automatically retract for tasks that have already passed, and uses regular blocks for the currently active task. That’s a neat way of highlighting the active task without much effort.

Peter doesn’t use templates for the portfolios, but leaves it up to the students how they want to present their learning evidence. They see the setting up of the portfolio as creative process and don’t want to narrow that down for the students. While his students submit written reflections, he likes giving feedback via video as he can point out the items that he’s referring to in the video more easily and can also show quickly how to do something in Mahara if needed.

What was especially interesting to learn is that private feedback is often sent via email rather than put on the page as private feedback. It seemed that the sending of mail was easier for that. Public feedback was left on the Mahara pages though.

PH Wien: Students

Reinhard gave an overview of the use of Mahara at PH Wien. As so many of us know, he did re-iterate the fact that the task design is crucial when working with portfolios and especially electronic portfolios. Many portfolios that he sees are just text or PDF documents uploaded to Mahara. That is just one step away from a paper portfolio. How can this be changed? Could one possibility be edubreak, which we’ll see later on?

Reinhard discusses portfolio use at PH Wien

Reinhard discusses portfolio use at PH Wien

Faculty do struggle with the vast amount of reflections especially when they have lots of students. Therefore, students, like at PH Niederösterreich, don’t reflect on everything, but are more discerning to decide which learning evidence should be reflected on, which shows growth the best rather than reflecting on everything. While they do want feedback from faculty, they can’t always get it due to the number of students in a class. They also work with peer feedback and students learn to find positive aspects so as not to point out only negative things.

Reinhard mentioned that sometimes portfolio reflections don’t convey the truth about what actually happened because students ask themselves what their instructor might want to hear or see. To overcome this behavior, changes in the culture around teaching and learning as well as giving feedback would be required.

A big challenge that PH Wien has is to find a good balance between “Sammelmappe”, i.e. storing of the evidence, and reflecting on evidence that illustrates learning well. A portfolio is not a backup / archive of all the things that students have done, but a selection of the best evidence that illustrates learning. These pieces should then be highlighted in reflections and discussed further. I think that other pieces of evidence could form additional context, but may not need their own reflection. That’s why I like the concept of the Mahara pages: You do not need to reflect / show only one piece of evidence, but you can place multiple pieces of evidence on a page and reflect on all of them at once and show the connections between them rather than looking at each evidence individually.

PH Wien: Schools

Being a teacher training college, PH Wien works closely with schools on a number of projects. One of which is KidZ (Klassenzimmer der Zukunft) Schaufenster [classrooms of the future showcase]. The project brings together best practice examples of how schools work with innovative learning and teaching methods as well as digital tools. In some cases, the schools and their portfolio implementations are also researched more closely. Some schools who haven’t had much experience showcasing their work in a portfolio received assistance by PH Wien in presenting their content.

Klaus tells us about the KidZ Schaufenster

Klaus tells us about the KidZ Schaufenster

It is great to learn that this showcase encouraged a number of schools to explore portfolio work themselves and to meet online to discuss their portfolios and how to improve them. Technology is still a hurdle for students and teachers at times. Some even argue for the old paper-based portfolio as they would have something tangible. However, the electronic portfolio gives them the freedom not having to carry crates of paper portfolios everywhere.

It will be interesting to find out more how technology can support students and teachers better, what improvements they’d like to see in particular and what impact these will have when implemented.

edubreak

Johannes gave a presentation on edubreak®campus, a social video learning platform developed in Germany. It is being used in the PREPARE project alongside Mahara. The platform, which is built on Drupal, allows teachers to prepare activities to which students upload videos. These videos can then be commented on in edubreak and the comments are overlayed on the video and easily accessible for viewing and further feedback by anybody who has access to the video.

Edubreak is already used successfully in trainings for table tennis coaches for example as the coaches-in-training can analyze the video, give feedback and learn how they can improve on their feedback.

We recorded my presentation on upcoming features in Mahara, our big project ideas, and then uploaded the recording to edubreak. Below, you can see what comments look like in the platform using the new HTML5 player.

Example of comments on an Edubreak video

Example of comments on an edubreak video

By using a video platform on which learners and teachers can engage in providing each other specific feedback, it is hoped that multimedia evidence becomes more accessible and that giving feedback on it becomes easier as viewers can jump to the individual feedback directly. When trialled at PH Wien, initially only a few students were willing to be video recorded and then have their videos made available for the tasks. However, once other students saw the power of reviewing their actions, they all took the opportunity to be filmed and then viewed their videos in edubreak.

Day 2

At the heart of the PREPARE project is the promotion of reflective practice in the training of teachers using ePortfolios. By combining edubreak with Mahara, the researchers are exploring how a video and reflective learning environment can foster competencies in the areas of reflection and teaching as well as transfer and exchange of knowledge and practical experiences by analyzing the learning outcomes closely through a learning analytics lens and making improvements in the learning environment accordingly.

On Day 2 of our get together, we explored how edubreak and Mahara could be used in conjunction by establishing personas as well as looking at technical implications of the integration such as authentication, transfer or linking of content and displaying videos and their comments in Mahara to make the experience as seamless as possible for the learners and also teachers.

The project team has more thinking to do and also trialling on how best to achieve the work flow and what is possible with web services between Mahara and edubreak. This may be one of the first projects to use the new connection manager that will be available in the next version of Mahara. Since it has already been merged into the master code, the team can start looking into it and give us feedback on its usability. The connection manager shall make hooking up external systems to Mahara easier so that they do not need to be configured in the code, but via a web interface.

I enjoyed both days very much as they gave me the opportunity to meet researchers in the area of ePortfolio practices, discuss their projects with them, learn about what goes well and what could be improved, as well as discuss the integration with edubreak, a social video platform, that can be beneficial for student learning and reflecting by them being able to easily annotate video content. As usual, time flew by very quickly, and after two days it was time to say goodbye to a great team whose progress I will follow. Seeing more integration projects develop will give us more insight into how well various systems can be connected to Mahara and what issues people encounter when they want to export or keep their portfolios offline and how we can circumvent them.

You can find the summaries of the two days from the perspective of Reinhard and Johannes in German.

16 July 2016

Jonathan Harker

British Museum

After a quick squizz in the Liberty and Schott Music shops, we ended up spending the rest of the day in the British Museum. The Rosetta Stone is the first thing you are presented with in the Egypt halls, yet we didn’t see it because it was surrounded by people which we mistook for simply a group of tourists. We wandered around admiring huge granite Babylonian lions, Assyrian friezes, marble chunks of the Parthenon and the Mausoleum of Halicarnassus, vaguely wondering where it was. Luckily there was a guided tour at 5pm that led us right to it. There was also red figure Greek pottery, bits of a marae, bronze age jewellery, coins from the Thames, cuneiform tablets, mediæval clocks and mechanisms, and Roman mosaics. So much history!

14 July 2016

Jonathan Harker

Salisbury Cathedral

Today we went to Andover to visit a special music shop, and then on to Salisbury Cathedral, via a quick rubber-neck at Stonehenge off the A303. We arrived in time for Evensong to hear the Leighton Magdalen service. The cathedral is huge, nearly 800 years old and beautiful, and the history of its construction inspired Follett’s novel, Pillars of the Earth. The Magna Carta document (or at least, one of the surviving four) hangs out the back on display in the 13th century chapter house, behind the cloister.

Salisbury Cathedral. Salisbury Cathedral cloister and cedars Rebekah was here!

Then we went to a nearby pub, called The Chapter House, for tea, then back to London.

Changing of the guard

IMG_5216
Is this not a quintessentially London image?

An hour on the No. 3 bus takes us to Westminster, alighting outside Parliament Buildings. From here we can have a look at Westminster Abbey, the new Supreme Court and then hang around until 11am for Big Ben to go “bong”.

IMG_5298
The gilded statue atop the Victoria Memorial, outside Buckingham Palace.

Then we leg it down Birdcage Walk to Buckingham Palace to join the ten thousand other tourists massed around the beautiful gilded Victoria Memorial, to watch the changing of the guard. One of the Royal Army bands played (among other things) arrangements of “Lucy In The Sky With Diamonds” and the theme from Game of Thrones. It’s also always a treat to hear cornet players wailing Birdland Maynard Ferguson style.

From there we had lunch at the Lodge Café by the Queen Elizabeth gates to Hyde Park, and strolled around Hyde Park for the afternoon. There are deck chairs out on the grass by Serpentine Lake, and they’re £2 an hour to sit in, which is… an odd thing to charge money for. I’d also forgotten that it’s 20p to use the loo!

A weeping elm in Hyde Park.
A weeping elm in Hyde Park.

We found a massive weeping elm that doubles as a house.

After that we went to Marble Arch and along Oxford Street to sort out SIM cards and suit fits for Hamish & Louise’s wedding, a bit of shopping and then to the pub on Kingly Street round the corner from Steve’s work, and catching up with some of the usual London suspects for curry.

10 July 2016

Kristina Hoeppner

face

0 degrees in hot summer

Who’d have thought you can find a 0 degrees Celcius spot in the heat of summer in Austria? Well actually, we didn’t have to look too far from Salzburg and found it in the Eisriesenwelt (ice giants world) in Werfen. We were looking for something to do with two children (aged 10 and 13) on Sunday, and a day in a hot city did not sound promising. Instead, we drove the 35 minutes to Werfen, up the mountains for 5 km and stood at the entrance to the Eisriesenwelt.

A short walk uphill, a 3-minute cable car ride and another short stroll uphill, and we were at the entrance of the cave, which is the biggest ice cave in the world. By that time alone we had climbed about 640 m. Most of that was via the extremely fast cable car though.

View from the entrance of the Eisriesenwelt

View from the entrance of the Eisriesenwelt

We put on jackets (and sturdy boots for everybody but me as I had left my hiking boots in New Zealand since I wasn’t expecting an excursion to the mountains) and were ready to go underground. Once it was our turn for the tour, every 4th or 5th person received an old-fashioned carbide lamp to light the way. Since we are in Europe, nobody worried about people mishandling them. There were no accidents and nobody burned off the pants of another person.

Entering the cave was a mission on its own as there was a wind blowing of at least 50 km/h once our tour guide opened the door. You really had to step through the door quickly and carefully, but as soon as you were inside, the wind was gone.

The cave itself is about 42 km long, but only the first kilometer is the actual ice cave. After that it is too warm for ice to stay. In winter, the door to the cave is left open so that icy wind can enter, and in the spring the snow melt trickles water through cracks forming the ice sculptures little by little in the way that stalacmites and stalactites are formed.

We went up 700 steps to reach the highest point in the cave (a walk of about 40 stories) that we could visit and down again 700 steps to return to our starting point. The biggest “piece” of ice in the cave is the ice wall and it’s about 25 m thick.

The ice inside the cave has a commanding presence, and it is fascinating to see that much ice inside the mountain and also wander through a huge cave like that. Since we only had the carbide lamps, the lighting was sparse but still plenty and added to the atmosphere. It was amazing to see the effort that had gone into exploring this cave and also creating the walkways and stairs (there was generally one for ascending and one for descending). Everyone was pretty quiet in the cave taking the surroundings in and being in awe of what we saw there.

The ice in the cave is mostly growing each year and the natural sculptures within it change all the time. What had been an elephant for the last many years is now slowly transforming into a dromedary. We could also see the growth of the ice over time clearly by looking at the layers in the ice. Our guide said though that in contrast to the rings in a tree, the layers in the ice can be added to or receed depending on the weather.

Since you can’t take photos inside the cave, you’ll have to go there for yourself to check it out.

02 December 2014

Andrew Ruthven

LCA2015 - Debian Miniconf & nz2015 Debian mini-DebConf

nz2015 mini-DebConf

Already attending linux.conf.au? Come a couple of days earlier and attend the mini-DebConf too! There will be a day of talks with a strong focus on the Debian project and a bug squashing day.

Debian Miniconf

After 5 years, the Debian Miniconf is back! Run as part of linux.conf.au 2015, this event will attract speakers talking on topics that suit the broader audience attending LCA. The Debian Miniconf has been one of the largest miniconfs in the history of linux.conf.au.

For more information about both these events which I'm organising, head over to: nz2015.mini.debconf.org!

25 August 2014

Dan Marsden

SCORM hot topics.

As a follow up from the GSOC post I thought it might be useful to mention a few things happening with SCORM at the moment.

There are currently approx 71 open issues related to SCORM in the Moodle tracker at the moment, of those 38 are classed as bugs/issues I should fix in stable branches at some point, 33 are issues that are really feature/improvement requests.

Issues about to be fixed and under development
MDL-46639 – External AICC packages not working correctly.
MDL-44548 – SCORM Repository auto-update not working.

Issues that are  high in my list of things to look at and I hope to look at sometime soon.
MDL-46961 – SCORM player not launching in Firefox when new window being used.
MDL-46782 – Re-entry of a scorm not using suspend_data or resuming itself should allow returning to the first sco that is not complete.
MDL-45949 – The TOC Tree isn’t quite working as it should after our conversion to YUI3 – it isn’t expanding/collapsing in a logical manner – could be a bit of work here to make this work in the right way.

Issues recently fixed in stable releases.
MDL-46940 – new window option not working when preview mode disabled.
MDL-46236 – Start new attempt option ignored if new window used.
MDL-45726 – incorrect handling of review mode.

New improvements you might not have noticed in 2.8 (not released yet)
MDL-35870 -Performance improvements to SCORM
MDL-37401 -SCORM auto-commit – allows Moodle to save data periodically even if the SCORM doesn’t call “commit”

New improvements you might not have noticed in 2.7:
MDL-28261 -Check for live internet connectivity while using SCORM – warns user if SCORM is unable to communicate with the LMS.
MDL-41476 – The SCORM spec defines a small amount of data that can be stored when using SCORM 1.2 packages, we have added a setting that allows you to disable this restriction within Moodle to allow larger amounts of data to be stored (you may need to modify your SCORM package to send more data to make this work.)

Thanks to Ian Wild, Martin Holden, Tony O’Neill, Peter Bowen, André Mendes, Matteo Scaramuccia, Ray Morris, Vignesh, Hansen Ler, Faisal Kaleem and many other people who have helped report/test and suggest fixes related to SCORM recently including the Moodle HQ Integration team (Eloy, Sam, Marina, Dan, Damyon, Rajesh) who have all been on the receiving end of reviewing some SCORM patches recently!

GSOC 2014 update

Another year of GSOC has just finished and Vignesh has done a great job helping us to improve a number of areas of SCORM!
I’m really glad to finally have some changes made to the JavaScript datamodel files as part of MDL-35870 – I’m hoping this will improve the performance of the SCORM player as the JavaScript can now be cached properly by the users browser rather than dynamically generating it using PHP.

Vignesh has made a number of general bug fixes to the SCORM code and has also tidied up the code in the 2.8 branch so that it now complies with Moodle’s coding guidelines.

These changes have involved almost every single file in the SCORM module and significant architectural changes have been made. We’ve done our best to avoid regresssions (thanks Ray for testing SCORM 2004) but due to the large number of changes (and the fact that we only have 1 behat test for SCORM) It would be really great if people could test the 2.8 branch with their SCORM content before release so we can pick up any other regressions that may have occurred.

Thanks heaps to Vignesh for his hard work on SCORM during GSOC – and kudos to Google for running a great program and providing the funding to help it happen!

10 July 2014

Andrew Ruthven

Cloud - in New Zealand!

I've spent a reasonable chunk of the past year working on a project we launched last month, Catalyst Cloud! It is using OpenStack with Ceph as the object store. It has taken a lot of work, and it is now very exciting seeing the level of interest there we're receiving about this new service!

The great part of this is that we can now offer private cloud services to our customers which provides all the flexibility that we've come to expect with the "cloud", but hosted in New Zealand by a New Zealand owned company so no concerns about jurisdiction of your data! Not only are we able to offer private cloud services on our OpenStack cluster(s), but we can also deploy OpenStack onto our customers own hardware using our ProdStack solution (I get to look directly at the Dashboard shown on that page, which is pretty cool).

Next up is deploying another OpenStack cluster in our new data centre (which is another project I'm working on). In the near future we also hope to start using Open Compute Project hardware for our clusters.

Dan Marsden

Goodbye Turnitin…

Time to say goodbye to the “Dan Marsden Turnitin plugin”… well almost!

Turnitin have done a pretty good job of developing a new plugin to replace the code that I have been working on since Moodle 1.5!

The new version of their plugin contains 3 components:

  1. A module (called turnitintool2) which contains the majority of the code for connecting to their new API and is a self-contained activity like their old “turnitintool” plugin
  2. A replacement plugin for mine (plagiarism_turnitin) which allows you to use plagiarism features within the existing Moodle Assignment, Workshop and forum modules.
  3. A new Moodle block that works with both the above plugins.

The Moodle.org Plugins database entry has been updated to replace my old code with the latest version from Turnitin, we have a number of clients at Catalyst using the new plugin and the migration has mostly gone ok so far – there are a few minor differences between my plugin and the new version from Turnitin so I encourage everyone to test the upgrade to the new version before running it on their production sites.

I’m encouraging most of our clients to update to the new plugin at the end of this year but I will continue to provide basic support for my version running on all Moodle versions up to Moodle 2.7 and my code continues to be available from my github repository here:
https://github.com/danmarsden/moodle-plagiarism_turnitin

Thanks to everyone who has helped in the past with the plugin I wrote – hopefully this new version from Turnitin will meet everyone’s needs!

31 October 2012

Chris Cormack

Signoff statistics for October 2012

Here are the signoff statistics for bugs in October 2012
  • Kyle M Hall- 24
  • Owen Leonard- 18
  • Chris Cormack- 15
  • Nicole C. Engard- 10
  • Mirko Tietgen- 9
  • Marc Véron- 6
  • Frédéric Demians- 5
  • Jared Camins-Esakov- 5
  • Magnus Enger- 4
  • Jonathan Druart- 4
  • M. de Rooy- 3
  • Melia Meggs- 3
  • wajasu- 2
  • Paul Poulain- 2
  • Fridolyn SOMERS- 2
  • Tomás Cohen Arazi- 2
  • Matthias Meusburger- 1
  • Katrin Fischer- 1
  • Julian Maurice- 1
  • Koha Team Lyon 3- 1
  • Mason James- 1
  • Elliott Davis- 1
  • mathieu saby- 1
  • Robin Sheat- 1

16 October 2012

Chris Cormack

Unsung heroes of Koha 26 – The Ada Lovelace Day Edition

Darla Grediagin

Darla has been using Koha from 2006, for the Bering Strait School District in Alaska. This is pretty neat in itself, what is cooler is that as far as I know, they have never had a ‘Support Contract’. Doing things either by themselves or with the help of IT personnel as needed. One of Darla’s first blogposts that I read was about her struggles trying to install Debian on an Emac. I totally respect anyone who is trying to reclaim hardware from the darkside 🙂

Darla has presented on Koha at conferences, and maintains a blog that has useful information, including sections of what she would do differently. As well as some nice feel good bits like this, from April 2007

I know I had an entry titled this before, but I do love OSS programs.   Yesterday I mentioned that I would look at Pines because I like the tool it has to merge MARC records.  Today a Koha developer emailed me to let me know that he is working on this for Koha and it should be available soon.  I can’t imagine getting that kind of service from a vendor.

Hopefully she will be able to make it Kohacon13 in Reno, NV. It would be great to put a face to the email address 🙂

10 October 2012

Chris Cormack

New Release team for Koha 3.12

Last night on IRC the Koha Community elected a new release team, for the 3.12 release. Once again it is a nicely mixed team, there are 16 people involved, from  8 different countries (India, New Zealand, USA, Norway, Germany, France, Netherlands, Switzerland) and four of the 12 roles are filled by women.

The release team will be working super hard to bring you the best release of Koha yet, and you can help:

  • Reporting bugs
  • Testing bug fixes
  • Writing up enhancement requests
  • Using Koha
  • Sending cookies
  • Inventing time travel
  • Killing MARC
  • Winning the lottery and donating the proceeds to the trust to use for Koha work.

24 July 2012

Pass the Source

Google Recruiting

So, Google are recruiting again. From the open source community, obviously. It’s where to find all the good developers.

Here’s the suggestion I made on how they can really get in front of FOSS developers:

Hi [name]

Just a quick note to thank you for getting in touch of so many our
Catalyst IT staff, both here and in Australia, with job offers. It comes
across as a real compliment to our company that the folks that work here
are considered worthy of Google’s attention.

One thing about most of our staff is that they *love* open source. Can I
suggest, therefore, that one of the best ways for Google to demonstrate
its commitment to FOSS and FOSS developers this year would be to be a
sponsor of the NZ Open Source Awards. These have been very successful at
celebrating and recognising the achievements of FOSS developers,
projects and users. This year there is even an “Open Science” category.

Google has been a past sponsor of the event and it would be good to see
you commit to it again.

For more information see:

http://www.nzosa.org.nz/

Many thanks
Don

09 July 2012

Andrew Caudwell

Inventing On Principle Applied to Shader Editing

Recently I have been playing around with GLSL Sanbox (github), a what-you-see-is-what-you-get shader editor that runs in any WebGL capable browser (such as Firefox, Chrome and Safari). It gives you a transparent editor pane in the foreground and the resulting compiled fragment shader rendered behind it. Code is recompiled dynamically as the code changes. The latest version even has syntax and error highlighting, even bracket matching.

There have been a few other Webgl based shader editors like this in the past such as Shader Toy by Iñigo Quílez (aka IQ of Demo Scene group RGBA) and his more recent (though I believe unpublished) editor used in his fascinating live coding videos.

Finished compositions are published to a gallery with the source code attached, and can be ‘forked’ to create additional works. Generally the author will leave their twitter account name in the source code.

I have been trying to get to grips with some more advanced raycasting concepts, and being able to code something up in sandbox and see the effect of every change is immensely useful.

Below are a bunch of my GLSL sandbox creations (batman symbol added by @emackey):

    

    

GLSL Sandbox is just the latest example of the merit of software development tools that provide immediate feedback, and highlights the major advantages of scripting languages have over heavy compiled languages with long build and linking times that make experimentation costly and tedious. Inventing on Principle, a presentation by Bret Victor, is a great introduction to this topic.

I would really like a save draft button that saves shaders locally so I have some place to save things that are a work in progress, I might have to look at how I can add this.

Update: Fixed links to point at glslsandbox.com.

05 June 2012

Pass the Source

Wellington City Council Verbal Submission

I made the following submission on the Council’s Draft Long Term Plan. Some of this related to FLOSS. This was a 3 minute slot with 2 minutes for questions from the councillors.

Introduction

I have been a Wellington inhabitant for 22 years and am a business owner. We employ about 140 staff in Wellington, with offices in Christchurch, Sydney, Brisbane and the UK. I am also co-chair of NZRise which represents NZ owned IT businesses.

I have 3 Points to make in 3 minutes.

1. The Long Term plan lacks vision and is a plan for stagnation and erosion

It focuses on selling assets, such as community halls and council operations and postponing investments. On reducing public services such as libraries and museums and increasing user costs. This will not create a city where “talent wants to live”. With this plan who would have thought the citizens of the city had elected a Green Mayor?

Money speaks louder than words. Both borrowing levels and proposed rate increases are minimal and show a lack of investment in the city, its inhabitants and our future.

My company is about to open an office in Auckland. A manager was recently surveying staff about team allocation and noted, as an aside, that between 10 and 20 Wellington staff would move to Auckland given the opportunity. We are not simply competing with Australia for hearts and minds, we are competing with Auckland whose plans for investment are much higher than our own.

2. Show faith in local companies

The best way to encourage economic growth is to show faith in the talent that actually lives here and pays your rates. This means making sure the council staff have a strong direction and mandate to procure locally. In particular the procurement process needs to be overhauled to make sure it does not exclude SME’s (our backbone) from bidding for work (see this NZCS story). It needs to be streamlined, transparent and efficient.

A way of achieving local company participation in this is through disaggregation – the breaking up large-scale initiatives into smaller, more manageable components. For the following reasons:

  • It improves project success rates, which helps the public sector be more effective.
  • It reduces project cost, which benefits the taxpayers.
  • It invites small business, which stimulates the economy.

3. Smart cities are open source cities

Use open source software as the default.

It has been clear for a long time that open source software is the most cost effective way to deliver IT services. It works for Amazon, Facebook, Red Hat and Google and just about every major Silicon Valley success since the advent of the internet. Open source drives the internet and these companies because it has an infinitely scalable licensing and model – free. Studies, such as the one I have here from the London School of Economics, show the cost effectiveness and innovation that comes with open source.

It pains me to hear about proposals to save money by reducing libraries hours and increasing fees, when the amount of money being saved is less than the annual software licence fees our libraries pay, when world beating free alternatives exist.

This has to change, looking round the globe it is the visionary and successful local councils that are mandating the use of FLOSS, from Munich to Vancouver to Raleigh NC to Paris to San Francisco.

As well as saving money, open source brings a state of mind. That is:

  • Willingness to share and collaborate
  • Willingness to receive information
  • The right attitude to be innovative, creative, and try new things

Thank you. There should now be 2 minutes left for questions.

05 January 2012

Pass the Source

The Real Tablet Wars

tl;dr formally known as Executive Summary, Openness + Good Taste Wins

Gosh, it’s been a while. But this site is not dead. Just been distracted by indenti.ca and twitter.

I was going to write about Apple, again. A result of unexpected and unwelcome exposure to an iPad over the Christmas Holidays. But then I read Jethro Carr’s excellent post where he describes trying to build the Android OS from Google’s open source code base. He quite mercilessly exposes the lack of “open” in some key areas of that platform.

It is more useful to look at the topic as an issue of “open” vs “closed” where iPad is one example of the latter. But, increasingly, Android platforms are beginning to display similar inane closed attributes – to the disadvantage of users.

Part of my summer break was spent helping out at the premier junior sailing regatta in the world, this year held in Napier, NZ. Catalyst, as a sponsor, has built and is hosting the official website.

I had expected to swan around, sunbathing, drinking cocktails and soaking up some atmosphere. Instead a last minute request for a new “live” blogging section had me blundering around Joomla and all sorts of other technology with which I am happily unfamiliar. Days and nightmares of iPads, Windows, wireless hotspots and offshore GSM coverage.

The plan was simple, the specialist blogger, himself a world renown sailor, would take his tablet device out on the water on the spectator boat. From there he would watch and blog starts, racing, finishes and anguished reactions from parents (if there is one thing that unites races and nationalities, it is parental anguish over sporting achievement).

We had a problem in that the web browser on the tablet didn’t work with the web based text editor used in the Joomla CMS. That had me scurrying around for a replacement to the tinyMCE plugin, just the most common browser based editing tool. But a quick scan around various forums showed me that the alternative editors were not a solution and that the real issue was a bug with the client browser.

“No problem”, I thought. “Let’s install Firefox, I know that works”.

But no, Firefox is not available to iPad users  and Apple likes to “protect” its users by only tightly controlling whose applications are allowed to run on the tablet. Ok, what about Chrome? Same deal. You *have* to use Apple’s own buggy browser, it’s for your own good.

Someone suggested that the iPad’s operating system we were using needed upgrading and the new version might have a fixed browser. No, we couldn’t do that because we didn’t have Apple’s music playing software, iTunes, on a PC. Fortunately Vodafone were also a sponsor and not only did they download an upgrade they had iTunes handy. Only problem, the upgrade wiped all the apps that our blogger and his family had previously bought and installed.

Er, and the upgrade failed to fix the problem. One day gone.

So a laptop was press ganged into action, which, in the end was a blessing because other trials later showed that typing blogs fast, on an ocean swell, is very hard without a real keyboard. All those people pushing tablets at schools, keep in mind it is good to have our children *write* stuff, often.

The point of this post is not really to bag Apple, but to bag the mentality that stops people using their own devices in ways that help them through the day. I only wanted to try a different browser to Safari, not an unusual thing to do. Someone else might want to try out a useful little application a friend has written for them, but that wouldn’t be allowed.

But the worst aspect of this is that because of Apple’s success in creating well designed gadgets other companies have decided that “closed” is also the correct approach to take with their products. This is crazy. It was an open platform, Linux Kernel with Android, that allowed them to compete with Apple in the first place and there is no doubt that when given a choice, choice is what people want – assuming “taste” requirements are met.

Other things being equal*, who is going to chose a platform where the company that sold you a neat little gadget controls all the things you do on it? But there is a strong trend by manufacturers such as Samsung, and even Linux distributions, such asUbuntu, to start placing restrictions on their clients and users. To decide for all of us how we should behave and operate *our* equipment.

The explosive success of the personal computer was that it was *personal*. It was your own productivity, life enhancing device. And the explosive success of DOS and Windows was that, with some notable exceptions, Microsoft didn’t try and stop users installing third party applications. The dance monkey boy video is funny, but the truth is that Microsoft did want “developers, developers, developers, developers” using its platforms because, at the time, it knew it didn’t know everything.

Apple, Android handset manufacturers and even Canonical (Ubuntu) are falling into the trap of not knowing that there is stuff they don’t know and they will probably never know. Similar charges are now being made about Facebook and Twitter. The really useful devices and software will be coming from companies and individuals who realise that whilst most of what we all do is the same as what everyone else does, it is the stuff that we do differently that makes us unique and that we need to control and manage for ourselves. Allow us do that, with taste, and you’ll be a winner.

PS I should also say “thanks” fellow sponsors Chris Devine and Devine Computing for just making stuff work.

* I know all is not equal. Apple’s competitive advantage it “has taste” but not in its restrictions.

18 May 2011

Andrew Caudwell

Show Your True Colours

This last week saw the release of fairly significant update to Gource – replacing the out dated, 3DFX-era rendering code, with something a bit more modern, utilizing more recent OpenGL features like GLSL pixel shaders and VBOs.

A lot of the improvements are under the hood, but the first thing you’ll probably notice is the elimination of banding artifacts in Bloom, the illuminated fog Gource places around directories. This effect is pretty tough on the ‘colour space’ of so called Truecolor, the maximum colour depth on consumer monitors and display devices, which provides 256 different shades of grey to play with.

When you render a gradient across the screen, there are 3 or 4 times more pixels than there are shades of each colour, producing visible ‘bands’ of the same shade. If multiple gradients like this get blended together, as happens with bloom, you simply run out of ‘in between’ colours and the issue becomes more exaggerated, as seen below (contrast adjusted for emphasis):

        

Those aren’t compression artifacts you’re seeing!

Gource now uses colour diffusion to combat this problem. Instead of sampling the exact gradient of bloom for the distance of a pixel from the centre of a directory, we take a fuzzy sample in that vicinity instead. When zoomed in, you can see the picture is now slightly noisy, but the banding is completely eliminated. Viewed at the intended resolution, you can’t really see the trickery going on – in fact the effect even seems somewhat more natural, a bit closer to how light bouncing off particles of mist would actually behave.

        

The other improvement is speed – everything is now drawn with VBOs, large batches of objects geometry passed to the GPU in as few shipments as possible, eliminating CPU and IO bottle necks. Shadows cast by files and users are now done in a second pass on GPU using the same geometry as used for the lit pass – making them really cheap compared to before when we effectively wore the cost of having to draw the whole scene twice.

Text is now drawn in single pass, including shadows, using some fragment shader magic (take two samples of the font texture, offset by 1-by-1 pixels, blend appropriately). Given the ridiculous amount of file, user and directory names Gource draws at once with some projects (Linux Kernel Git import commit, I’m looking at you), doing half as much work there makes a big difference.

06 October 2010

Andrew Caudwell

New Zealand Open Source Awards

I discovered today that Gource is a finalist in the Contributor category for the NZOSA awards. Exciting stuff! A full list of nominations is here.

I’m currently taking a working holiday to make some progress on a short film presentation of Gource for the Onward!.

Update: here’s the video presented at Onward!:

Craig Anslow presented the video on my behalf (thanks again Craig!), and we did a short Q/A over Skype afterwards. The music in the video is Aksjomat przemijania (Axiom of going by) by Dieter Werner. I suggest checking out his other work!

14 August 2009

Piers Harding

Auth SAML 2.0 for Mahara

Following on from the SAML 2.0 work that I've done recently for Moodle, I thought it was useful to do the same for the Mahara ePortfolio service, while I was in the same space. Details of the first release can be found here, with tested version for both trunk, and 1.1_STABLE.

02 August 2009

Piers Harding

Moodle and SAML 2.0 Web SSO

Of late I have been doing a lot of SSO integration work for the NZ Ministry of Education, and during this time I came across an excellent project FEIDE. One of the off shoots of this has been the development of a high quality PHP library for SAML 2.0 Web SSO - SimpleSAMLPHP.

For Moodle integration, Erlend Strømsvik of Ny Media AS, developed an authentication plugin, which I've made a number of changes to around configuration options, and Moodle session integration. This has now been documented and added to Moodle Contrib to give it better visibility to the Moodle community at large. Documentation is here and the contrib entry is here.

27 June 2009

Piers Harding

Perl sapnwrfc 0.30

I doing some work for a client recently, I got the opportunity to do some major performance work on sapnwrfc for Perl. The net result is that a number of memory leaks, mainly of Perl values not going out of scope properly, have been fixed.

Additionally, I've had some time to put together a proper cookbook style set of examples in the sapnwrfc-cookbook. These examples, while specifically for Perl, are almost identical for sapnwrfc for Python, Ruby, and PHP too.