Macrotone Blogs

Macrotone blogs upon Joomla, our products and other matters.

Problems with Windows 7 SP1 installation

windows7The question of quite how one is perceived as an expert in all things computing when one knows a little about one aspect of IT, is one that I will leave for those much more knowledgeable than I.

The other week I was presented with a PC that was reported as ‘crashing’ whilst in use.

I must admit I never managed to find it  crashing on me whilst I was looking it over, but there were a number of things that didn’t look quite right, as is usual when presented with a machine with ‘problems’.  There was a lot of ‘temp files’, consuming disk space and it was split into two partitions, the second of which (the data partition D:) was virtually empty and the system partition C: was virtually full, so a repartitioning would be a good start.  The Windows registry revealed a lot of entries that were not being used and there were obviously a lot of updates missing of which the most obvious was SP1.  One proceeded to run virus scans and starts updating with the latest Microsoft fixes. However whilst most of the updates applied themselves without any issues. SP1 would fail with an error message ‘0x800f081f’.

Searching the web suggested various solutions such as running a full disk check, installing the Windows Update Readiness Tool etc., but none of these made the slightest difference to the symptoms, even though there were a few disk errors detected and corrected.  As each solution involved quite a long time to complete, especially the disk check this was not going to be a quick resolution.

The Microsoft web site didn’t throw up any possible solution, but surely after all this time, I couldn’t be the first person to hit this problem? After several days spent trying the suggestions and after much searching I eventually found this link. Although it was concerning Windows 2008 R2 it was still applicable. The symptoms described almost exactly match what we were experiencing. Even the screenshots reflected the symptoms we observed, although the event log display was slightly different.  With this closeness we tried the suggestion.

The removal of the patch KB976932 with the Deployment Image Services and Management Tool, took over an hour and it was not always immediately obvious that it was being removed, or even doing anything at all, but one resisted the temptation to ‘tinker’.  Once removed the next attempt to install the SP1 upgrade immediately started working.

Over an hour later after a reboot we could then check for any further updates. As you may guess there were nearly another 100 patches to apply, so we were not complete yet, but at least we were over our major hurdle and we were almost ready to finish our tests and give the machine back to its owners.

Problems with a SEF component

We are currently trying to track down a strange problem with an installed SEF component.  The problem fortunately only occurs on our Development site but it has some very strange symptoms.

First we created our development site with a backup of our live environment.  Everything in the back end works fine but in the front end some pages display completely blank, with no tags what so ever.   Fine one would say, there must be a PHP error, (although why this should only occur on a restored system rather than the live site is another question), so we turn on ‘Site Error’ reporting to Maximum (or Development) and it makes no difference to the front end displays they still shown blank pages.

Perhaps a cache problem, but no, clearing all the caches, in the system, browser etc., still makes no difference.  Nothing is reported in system or error logs what so ever.

Then a break though, we disable our SEF component and the pages display OK. So if must be something with the SEF component!  We reinstall an older version of the SEF component and it all works perfectly.  Re-install the current version of the SEF component and the blank pages re-occur.  Re-install the ‘older’ version and the pages are fine.

At this point we report the problem to the component author’s support forum.

Now frustration starts to kick in.  The support from the vendor says that there isn’t a problem, since there is no PHP error, and it doesn’t occur on our live site!  Exactly what sort of support is it they are offering? And this is a component for which we pay for support!  I will withhold the name of the vendor for the moment, perhaps they will improve, and it might be a problem at my end, although it is not looking likely.

I am starting to hate SEF with a vengeance, and this is just another straw that is ‘starting to break the camels back’.

At least we have a mechanism to enable us to continue testing our components for a forth coming release, but we really need to resolve this at some time, so it will be revisited when we have some more time.  Watch this space.

Converting Joomla Discussions Forum to Kunena revisited

kunenaWe previously wrote about converting a Discussions Forum to Kunena and it was now time to revisit the topic as we prepared to perform the task again and this time make the final step to make it live on our site. Since then the Kunena version has moved on to 3.0.3 so we were expecting a few changes.

We must also take the opportunity to state that our Discussions forum did not make use of images/attachments, hence it was not necessary to consider these in the migration. There was also a know opportunity with the Discussions component in that although new user were synchronised between Joomla and the component, when users were removed from teh system for any reason, their details remained within the component. It was thus necessary to run an additional SQL script to clean out these old entries.

The installation of Kunena 2.0.4 and the Importer worked fine, as did the import of the Discussions items (posts and categories). Unfortunately the ‘live update’ to version Kunena 3.0.3 gave us a blank screen, so we resorted to the standard update mechanism which worked fine. (Perhaps the live update implementation needs some work?)

Then we started running our SQL scripts (See previous post for details, which are didn’t require any changes):

  1. Create the  view.
  2. Updated the topics table. We observed the message ‘1364: Field 'params' doesn't have a default value’.
  3. Update categories number of topics count. We saw a number of messages ‘1048: Column 'numTopics' cannot be null’.
  4. Update the last_topic_id, last_post_id and last_post_time fields.  Again we saw a number of ‘1048:Column xxxxx cannot be null’ messages. We were not too worried about these messages as they reflect the empty categories in our original Discussions source tables.
  5. We then updated the category hits.
  6. We then recalculated the statistics within the Kunena Forum Tools.
  7. As mentioned above, we run an additional script to remove 'old' users from our Kunena users table.

delete from #__kunena_users
where userid NOT IN (select id from #__users);

Note: There are several fields such as the number of views of a topic etc. which are not populated because this information is not retained by our source forum and we could not begin to guess what might be appropriate, so better to leave them empty. They will populate themselves over time and we are not too interested in how many people had read them.

Now we can either start configuring the component on our development site, or we can export our tables ready for loading into our live site and after loading configure the forum there. We decided upon the later option.

Inspecting the database tables if was apparent that a number of the tables were empty (since this was effectively a new forum installation) so only a small number needed to be exported and imported on our live site.  The list of tables was as follows:

#__kunena_aliases, #__kunena_categories, #_kunena_configuration, #__kunena_messages, #__kunena_messages_text, #__kunena_topics, #__kunena_user_topics, #__kunena_users, #_kunena_version

On our target (live) site we only need to install the latest Kunena version (3.0.3) and import the data we exported above.

We now disable the use of sh404SEF for the Kunena forum. Basically we have found it more trouble than it is worth, and the Forum entries are fine without the need to use the SEF translations in sh404SEF.

We now enable the Forum menu item, which is the menu link to the Kunena menu item, and after clearing out system cache, we are ready to see how it is looking.

Looking at the front end of the site we can enter our forum and see our posts etc., so all is basically sound.

The next step is to install an anti spam component for the Forum. We have chosen R-Antispam v1.3.0 by Ratmil, which uses a Bayesian algorithm to determine whether the message is spam or not. Being Bayesian it learns over time and becomes more accurate the longer it is used.

Our configuration included adding spam check API keys, changing the default Kunena menu tab, etc.  We also had to change our category headers slightly since Kunena didn’t like html in the category description.

Once done we made the new ‘forum’ available to users and disabled the ‘old’ forum.  Note the ‘disabled’ since we will monitor it over a few days/weeks before we finally ditch it altogether.

Addendum: Have to also change the SEF entries that refer to the 'old' forum entries to redirect to the new forun equivalent. Also search out 'smart search/finder' entries as well.  Note that Kunena does not currently install a 'smart search' plugin even though it is present in the installation file. See Kunena web site forum for the rationale for this.

Web Site Security

b2ap3 icon joomlaJust read a short article in the December issue of the Joomla Community Magazine titled ‘Ten Arguments That Threaten the Security of Your Website’ that is well worth reading.  It applies equally well to any web site but obviously the emphasis is upon Joomla.

The point about always keeping your website up to date with the latest patches in particular is one that I usually have to use every week, when looking at reported problems our component users are experiencing.   Usually the Joomla version is several if not many versions behind current.  Perhaps one might miss or not have time to always be upon the latest release but there is really no valid reason for being, in some cases over a year behind.  It is just asking for trouble.

I recommend it for a worth while read, it is reasonably short, but quite concise, and unfortunately so true in many respects.

Problems with ReCaptcha display

ReCaptchaWe have been seeing a strange problem on our site where the ‘ReCaptcha’ challenge was not always being displayed when it should have been. Most notably this is/was seen on the site registration page, but was not just restricted to that page. A complete page refresh often resolves the problem and shows the Captcha block.

We today saw that a change is proposed for the next Joomla update. It is mentioned in this post from OSTraining, which in turn refers to the fix itself which is extracted from this joomla.org doc.

The basic problem seems to be that Google changed the URL’s of the Recaptcha API location, which doesn’t completely explain what we are/were seeing, but implementing the fix cannot do any harm and may even resolve the problem.

We will watch it for a while and see whether it completely solves the problem, but recommend it is implemented on your sites if you are experiencing ReCaptcha problems.  It does require FTP or SSH/Telnet access to change the plugin code.

Update 29/11/2013: Doesn't solve the display problem, so still investigating.

Implementing Online Storage - DropBox

b2ap3 icon dropboxWe have been investigating the use of ‘Cloud services’ in particular for backup and synchronising devices and discovered just how easy it was to use.   One attraction is that there is a ‘free’ no cost option, and there appears to be a wide variety of providers.  We have used Amazon Cloud Drive, Google Drive and also Microsoft’s SkyDrive but they didn’t easily integrate into all of our environments and we wanted to automate the use of the service.

Before we jump straight in, one often asked question is what is online storage?  The answer is of course, that online storage, or "cloud" services as they're also known, it that it is ‘storage space’ help somewhere in the ‘internet cloud’ that appears (often) as a virtual hard drive that's shown on your desktop and linked directly via the internet to the suppliers online space.  Exactly where it is stored is not really important, just that it is accessible.

They are easy to use, as one just opens the folder on the desktop and copies or pastes a file (or files) whether they be documents, music files, photos etc., and the files get synchronised across to the ‘online’ space.  If one has multiple devices (or machines) then each device, with the ‘providers’ software installed can see and access to files.

It is also possible to ‘share’ the files with other users as well,  once they've installed the same service.

Security

Data security is the number one concern since one is relying on the service to keep your files and documents secure.  If your account is hacked, your files are immediately available. However, there are several things you can do to prevent this:

  • Frequently change your passwords and don't use the same passwords as your email accounts. As a lot of these services require you to use your email address as your ID, it makes it easier for the bad guys to crack your password.
  • Use some encryption upon the files which is encoded with a password for sensitive files stored on the ‘online service.

And of course the ‘old’ standards still apply:

  • Install (free?) antivirus software and malware software, and keep it up-to-date.
  • Avoid opening any links or attachments that could be potential security risks.
  • Beware of phishing emails and any messages from unknown senders that request your bank details.

Note: Other ‘online storage’ users usually can't see your private files unless one deliberately invites them or places the files in the "public" (or shared) folder.  As expected everything in your public folder is, by definition, accessible to anyone.

As we said at the beginning we wanted to specifically concentrate on the ‘offsite’ storing of ‘backups’, in particular our web server.  Fortunately we make use of Akeeba Backup Pro, which allows easy integration into a number of providers.  We chose to look specifically at DropBox which since its launch in 2008 has become well know with techies, and has become one of the biggest names in online storage. It is also available upon a number of devices and has a simple, user-friendly interface. It can be downloaded on to PCs, Macs, iPads, iPhones, BlackBerrys and Android devices.  It also has an online version of the service, which you can use on any PC where Dropbox is not installed - just log in online.  IT is this flexibility that particularly appeals.

It initially comes with 2GB of space, but this can be boosted by introducing ‘new’ friends to Dropbox.  With every new referral, you'll both gain an extra 500MB, up to a maximum of 18GB. If more is still required a monthly subscription can be added to the account.  This was more than suitable for our requirements so we read the documentation on the Akeeba site and proceeded to set up Akeeba Backup to create a backup and ran a test.  It worked perfectly, so much so that we thought we had made a mistake, but the site backup was in the Dropbox account. The automation was then set up using a cron ‘daemon’ on our web site and left to see how it performed overnight.  Checking the following day showed the backups were working well.

Next we looked at the synchronisation process between devices.  The only snag we found was that some of our devices were a ‘little long in the tooth’ and the Dropbox software required ‘up to date’ operating system versions, which were of course not available for  some of the hardware.  Apart from that the processes work well, with only the expected delays caused by the network performance.

The next thing we looked at was synchronising with our NAS, (again another device, albeit one with certain characteristics). Again installing the software package upon the NAS went smoothly just as explained in the NAS documentation., and it then automatically created a folder on the NAS and saved all the files onto the NAS from Dropbox.

So we achieved what we set out to do.  Automatically created a web site backup (via cron), and transferred it to the ‘cloud Dropbox service’. Then on our NAS box it automatically synchronised the Dropbox backup files to out local NAS storage. Job done.

All in all it was as ‘easy as pie’ and we recommend it.  In fact we wonder why we didn’t do it before, it just ‘worked’!

It is easy to see how beneficial this service could be to someone at or going to University who wanted to preserve their work in the unlikely event  of an accident or possible theft of their personal computer.

Download it now: Dropbox

Chinese Domain Name Scam

We received the following email yesterday and were immediately suspicious and did a search on the Web to see what we could find.  We immediately hit upon this blog article which matched what we had received.  Reading the blog post we could see that although our email had a nice looking logo and a slightly different registry name it was the same thing just dressed up a little differently.  The General manager has also got a new first name, perhaps he is the brother of the previous incumbents?

Looking a little further we wonder why an Office Supplies company would want to give IT consultancy services?

For those interested the blog article, describes it very well and we will not repeat it here.  Suffice to say, I also do not consider it worth while going into what I will be doing with the emails.

Just another example of the types of scams that we see so often these days!

=== Email follows ===========

(Please forward this to your CEO, because this is urgent. Thanks)

We are a Network Service Company which is the domain name registration center in Shanghai, China. On Nov 26, 2013, we received an application from Huamai Ltd requested "macrotoneconsulting" as their internet keyword and China (CN) domain names. But after checking it, we find this name conflict with your company name or trademark. In order to deal with this matter better, it's necessary to send email to you and confirm whether this company is your distributor or business partner in China?

Kind regards
Scott Zhang

***************************************************

Scott Zhang
General Manager 
Shanghai Office (Headquarters)
Tel: +86-21-6191-8696
Mobile: +86-182-2195-1605
Fax: +86-21-6191-8697

 cnregistry

============ End of Email =============

Tags:

Design Criteria for processing emailed issues

We have been busy with a forthcoming feature for our Issue Tracker, which is the ability to receive emails with details of new issues or emails containing updates to previously raised issues.

After much thought and consideration the following design criteria are  deemed necessary in the email issues functionality. They are included in no specific order of
priority.

Continue reading

Problems getting a front end wget/curl cron working.

joomlaIn completing the testing of our ‘cron’ addition to a Joomla component we experienced a small problem, which is worthy of noting especially as we have not seen any mention of the problem anywhere else on the web.

The problem manifested itself as the apparent ‘inactivity’ when a front end web page was accessed via wget, curl or lynx.  This was very puzzling since accessing via the usual web browser appeared to be working.  After much testing we discovered that wget, curl etc. were indeed accessing the page but the underlying actions that the page was intended to run were not occurring.

To cut a long story short, we discovered that the pages were being served up by the ‘page cache’.  In the normal course of events this is what one wants to occur, however with a ‘cron’ page we want the underlying actions to occur.  It was necessary to disable the caching of the specific accessed page and this completely resolved the problem.

What was interesting was that this specific problem was not mentioned on any sites on the web, despite extensive searches. Either everyone else is aware of the problem, possible but unlikely, or equally unlikely no-one else uses page caching.

As part of our investigations we also tried out a well known ‘free cron’ service.  This didn’t help us resolve our problem, but did make us aware of possible ‘redirections’ that may be caused by the use of ‘SEF’ components on a web site. It was necessary to specify the ‘redirection URL’ rather than the ‘original URL’ to the service. Not a specific problem but one of those things to be aware of. 

Problems installing a Joomla component

joomlaWe met a very strange problem when we had to re-install a third party Joomla component.  We will not go into the details of why we had to do a fresh re-install, sufficient to say it was to try and fix an ‘opportunity for improvement’.  We couldn’t uninstall using the standard Joomla mechanism because we wanted to retain all the underlying database tables used by the component so did a manual uninstall instead.

The component was quite large but we extracted all the files and placed them in the tmp directory, so it was easy to reinstall until we determined the cause of the problem.  [We encountered a few MySQL Error 2006 – DB Server has gone away’ messages, but apart from being annoying were not really causing a problem.]

The component ‘spun’ its icon and then presented us with the following messages:

"Component Install: DB function reports no errors"
"Error installing component"

In addition an entry was make in the #__extensions table but no files are ever installed.  It is impossible to remove it via the Joomla ‘Manage’ feature and all one can do is remove the entries from the table via phpadmin (or similar).

Attempts to re-install again, are repeatable and every time a new entry was inserted into #__extensions.  If one tried several times, several rows were created.

Then we found this problem report, which had all the symptoms we were seeing.  It was raised against Joomla 1.6 but was still very useful. The clue was the mention of the #__assets table and sure enough we found a single line entry in the #__assets table which we could then remove and the next attempt to reinstall worked perfectly.

Go To Top

Joomla! Debug Console

Session

Profile Information

Memory Usage

Database Queries