Drupal to Drupal - Content Migration

At some point you will feel the need for a new cleaner website, so you build your new Drupal site with cleaner content types, and you want to migrate your old content to the new website. Migrate module is the module for that, however, by itself you will not be able to do your migration, you have to build your migration classes on top of that module, so below are the migration steps.

We start with creating a new module that will depend on Migrate module. Create worchestra_legacy.info

name = "Worchestra Legacy"
description = ""
core = "7.x"
package = "Worchestra"

files[] = migrate/base.inc
files[] = migrate/articles.inc

Create worchestra_legacy.module and implement hook_migrate_api to tell Migrate module about our coming migration classes

/**
 * Implements hook_migrate_api().
 */
function worchestra_legacy_migrate_api() {
  return array(
    'api' => 2,
  );
}

In our .info file we declared our 3 new files that need creation inside a /migrate directory in worchesra_legacy module Lets start with base.inc, and define our old database location, you can also use base.inc to create any function that you will use later in the migration process.

abstract class WorchestraLegacyBaseMigration extends Migration {

  /**
   * Should be used for all source queries in order to connect to the right
   * database. It's required that this target database is defined in
   * settings.php.
   */
  var $legacy_db_key = 'worchestra_old';

  public function __construct($group = NULL) {
    parent::__construct($group);
  }

  /**
   * Modified version of db_select() that uses another database connection.
   */
  protected function db_select($table, $alias = NULL, $options = array()) {
    if (empty($options['target'])) {
      $options['target'] = 'default';
    }
    return Database::getConnection($options['target'], $this->legacy_db_key)->select($table, $alias, $options);
  }

}

Now that we defined our old database connection so our module will have access to the posts in the old database, we need to add the declaration in settings.php

$databases['worchestra_old']['default'] = array(
  'driver' => 'mysql',
  'database' => 'worchestra_old',
  'username' => 'mysql_username',
  'password' => 'mysql_password',
  'host' => 'the_host',
);

Now create our articles.inc that will handle the articles migration

// Create a class that extends our base class
class WorchestraLegacyArticlesMigration extends WorchestraLegacyBaseMigration {
  public function __construct() {
    parent::__construct();
    $this->description = t('Import articles');

    $this->map = new MigrateSQLMap(
      $this->machineName,
      array(
        'nid' => array(
          'type' => 'int',
          'not null' => TRUE,
          'description' => 'Node ID',
          'alias' => 'n',
        )
      ),
      MigrateDestinationNode::getKeySchema(),
      $this->legacy_db_key
    );

    $this->highwaterField = array(
      'name' => 'changed',
      'type' => 'int',
      'alias' => 'n',
    );

    // Query source tables.
    $query = $this->db_select('node', 'n');
    $query->innerJoin('field_revision_body_old', 'fr', 'n.vid = fr.revision_id');
    $query->innerJoin('field_data_field_tags_old', 'fdft', 'fdft.entity_id = n.nid');
    $query->leftJoin('node_counter', 'nc', 'nc.nid = n.nid');

    // Query source fields.
    $query
      ->fields('n', array('nid','uid', 'title', 'created', 'changed', 'status', 'language'))
      ->fields('fr', array('body_value', 'body_format'))
      ->fields('nc', array('totalcount', 'timestamp'));
    $query->addField('n', 'nid', 'path');
    // Source grouping.
    // We need to query the tags on each post, and concatenate them with a comma
    $query->addExpression('GROUP_CONCAT(DISTINCT field_tags_tid)', 'tags');
    $query->groupBy('n.nid');
    // Source conditions.
    $query->condition('n.type', 'blog');
    $query->orderBy('n.changed', 'DESC');
    $this->source = new MigrateSourceSQL($query);
    

    // Destination definition.
    // I want to maintain the old node URLs so I will maintain the node ID's
    $this->destination = new MigrateDestinationNode('article');
    $this->addFieldMapping('nid', 'nid');
    $this->addFieldMapping('n_uid', 'uid');
    $this->addFieldMapping('is_new')
         ->defaultValue(TRUE);
    // Mapping
    $this->addFieldMapping('title', 'title');
    $this->addFieldMapping('created', 'created');
    $this->addFieldMapping('changed', 'changed');
    $this->addFieldMapping('status', 'status');
    $this->addFieldMapping('body_new', 'body_value');
    $this->addFieldMapping('totalcount', 'totalcount');
    $this->addFieldMapping('body:format')
      ->defaultValue('full_html');        
    $this->addFieldMapping('field_topics_new', 'tags')
      ->separator(',')
      ->arguments(array('source_type' => 'tid'));

    $this->addFieldMapping('sticky')
      ->defaultValue(0);
    $this->addFieldMapping('promote')
      ->defaultValue(0);
    $this->addFieldMapping('revision')
      ->defaultValue(1);
    $this->addFieldMapping('comment')
      ->defaultValue(2);
  }

}

Now if you go to admin/content/migrate you will see our our node migration class resulted in a checkbox to migrate the nodes. A good point to note here is that in our article.inc class we can use a method called prepare() to do any processing, modification, or cleaning up on any field before saving it in the new database.

This is a simple example on how to do content migrations, you can extend it to migrate taxonomies, users, and file entities.

Drupal 8 - How building modules

Drupal 8 is in alpha now, so its a good time to dip your feet in it and notice the major differences Drupal 8 introduced after adopting some Symfony low-level components such as HttpKernel, Routing, EventDispatcher, DependencyInjection, and ClassLoader, for a higher level changes see Hector Iribarne’s presentation on Drupal 8 changes.

In this blog post I will try to build a D8 module and walk through the basic steps.

In this example series we will develop a module and build on it during the coming blog posts.

Lets start.

In brief, our module is basically is a greeting box, visitors come to the site and leave you their greetings and comments, you as an admin can look at these greetings and delete the ones that you do not like. So the basic required architecture is: - A table in the database to hold the greetings. (Model or schema API) - Menu callbacks (Routes) - Controllers to handle the logic.

A good point to note here is that custom and contributed modules/themes now live inside /themes or /modules in the root folder, but still you can place them inside sites/all/modules or sites/all/modules themes.

Like any module in Drupal 7, modules in Drupal 8 need a .info file but this time in a YML format, so we create our hello.info.yml inside our hello directory that we create.

name: Hello
type: module
description: "Hello Drupal 8"
version: 1.x
package: Custom
core: 8.x

Before enabling the module, lefts define our database model, just as in D7 place our code in hello.install

function hello_schema() {
  // Lets keep it basic here and only define the textarea
  $schema['names'] = array(
    'fields' => array(
      'name' => array(
        'type' => 'varchar',
        'length' => 40,
        'not null' => TRUE,
      )
    ),
    'primary key' => array('name'),
  );

  return $schema;
}

After enabling the module we will see the database table that we defined.

Menu callbacks in D7 used to live in .module file but not anymore (will come to that later) after introducing Symfony’s routing system, so lets create a file and call it hello.routing.yml and tell Drupal that we need a page that holds a form for people to greet us.

name_add:
  path: 'add/name'
  defaults:
    _form: '\Drupal\hello\Controller\AddName'
    _title: 'Add your name'
  requirements:
    _permission: 'access content'

Basically telling our module to define a menu route ‘add/name’ that takes its logic from AddName controller, page title should be “add your name”, and anyone with “access content” permission should be able to access this page.

As we said before, menu callbacks do not live in .module anymore, however, its a good practice to add a hook_menu() in the .module file

function hello_menu() {
  $items['add/name'] = array(
    'title' => 'Contact form categories',
    'description' => 'Create a system contact form and set up categories for the form to use.',
    // point to name_add route in hello.routing.yml
    'route_name' => 'name_add',
  );
  
  return $items;
}

Now we should create the controller AddName that will handle displaying the form in the page callback, so create a directory inside the module hello

mkdir -p lib/Drupal/hello/Controller
cd lib/Drupal/hello/Controller
vim AddName.php

In AddName.php

namespace Drupal\hello\Controller;
use Drupal\Core\Form\FormInterface;

class AddName implements FormInterface {
  function getFormID() {
    return 'add_name_form';
  }
  
  function buildForm(array $form, array &$form_state) {
    // The usual Drupal FAPI
    $form['name'] = array(
      '#type' => 'textfield',
      '#title' => t('Your name')
    );
    $form['actions'] = array(
      '#type' => 'actions'
    );
    $form['actions']['submit'] = array(
      '#type' => 'submit',
      '#value' => 'Add'
    );
    return $form;
  }
  
  function validateForm(array &$form, array &$form_state) {
    // Your validation code goes here
  }
  
  function submitForm(array &$form, array &$form_state) {
    $name = $form_state['values']['name'];
    drupal_set_message(t('Added %name as new participant.', array('%name' => $name)));
    // Remember: We should create a HelloStorage controller with the method add() later
    HelloStorage::add($name);
    // Redirect after form submission to add/name that or any other page that exists.
    $form_state['redirect'] = 'add/name';
  }
  
}

Now that we have our page add/name ready, lets create the HelloStorage controller before we try out the submission In the same directory that holds our controllers (lib/Drupal/hello/Controller) create the file HelloStorage.php

namespace Drupal\hello\Controller;

Class HelloStorage {
  
  static function getAll() {
    return db_query('SELECT name FROM {names}')->fetchCol();
  }
  
  static function add($name) {
    db_insert('names')->fields(array('name' => $name))->execute();
  }
  
  static function delete($name) {
    db_delete('names')->condition('name', $name)->execute();
  }

}
}

Alright, the form is ready to work.

Now lets create the admin page that will display the greetings. First we need to define our route in hello.routing.yml

name_list:
  path: 'list/names'
  defaults:
    _content: '\Drupal\hello\Controller\ListNames::content'
    _title: 'People'
  requirements:
    _permission: 'administer greetings'

The route needs a controller to handle it, so in the controllers directory we create the file ListNames.php and we define content() method

namespace Drupal\hello\Controller;

class ListNames {
  public function content() {
    $people = HelloStorage::getAll();
    $output = l('+ Add name', 'add/name');
    $output .= '<table><tr><td>Name</td><td>Delete</td></tr>';
    foreach ($people as $person) {
      $output .= '<tr><td>' . $person . '</td><td><a href="delete/' . $person . '">Delete</a></td></tr>';
    }
    $output .= '</table>';
    return $output;
  } 
}

As we can see from the code above, there is an l() function that create a delete link for the records in the database, that means we need to create a route for that, so in our routing file

name_delete:
  path: 'list/delete/{name}'
  defaults:
    _form: 'Drupal\hello\Controller\DeleteName'
    _title: 'Are you sure?'
  requirements:
    _permission: 'administer greetings'

Now we create our DeleteName controller

namespace Drupal\hello\Controller;
use Drupal\Core\Form\FormInterface;

class DeleteName implements FormInterface {
  function getFormID() {
    return 'delete_name_form';
  }
  
  function buildForm(array $form, array &$form_state) {
    // Just to confirm the deletion
    $form['actions']['submit']['yes'] = array(
      '#type' => 'submit',
      '#value' => 'Yes'
    );
    return $form;
  }
  
  function validateForm(array &$form, array &$form_state) {
    
  }

  function submitForm(array &$form, array &$form_state) {
    $name = arg(2);
    HelloStorage::delete($name);
    drupal_set_message(t('%name was deleted', array('%name' => $name)));
    $form_state['redirect'] = 'list/names';
  }

}

In the next post, we will build over our module to display the greetings in a block (Plugins) and define custom permissions.

How to .htaccess helped when migrated Drupal and old URLs became not found

This is indeed very simple, but I thought a lot of people faced this problem, so no harm to write about it.

In the old site I had Pathauto module installed to handle clean (or pretty) URLs patterns across different content types, so a post URL that was post/this-is-an-article became article/this-is-an-article which will result in a lot of ‘page not found’ and dead links in the site which extremely harms SEO, aside from annoying your visitors.

To solve this you can ask .htaccess file to handle the redirects from old URL patterns to new ones. So make sure mod_rewrite is enabled in your Apache settings (if you are using Apache), then inside your .htaccess file add the line below - but make sure to change it to reflect what you have.

RewriteRule ^post/(.*)$ http://worchestra.com/article/$1 [R=301,L]

Basically that will redirect old URLs to new ones, you can apply this to any URL pattern. Hope that helped.

Role Notify - in Drupal Module

I needed a functionality in Drupal that handles the notifications for users based on role on new content in the website, so I looked around and considered Notifications Module which is great, but adds much more than I really need for a simply notify module.

Role Notify is a light-weight module to work as an alternative module for simple notifications. It basically helps you notify all registered users of a selected role when a new content is published in the site.

Notifications can be selective with different settings on each content type. Users can choose whether to notify on the node level.

Features

• Per content type notification
• Per node notification
• Per role notification
• Token module support
• Handy role permissions administration
• Future release will extend this module to work with publishing workflows.

Difference to Rules Module

Rules module does help in a generic way. Role notify lets you choose whether you want to notify on a node and and not to notify on another from the same type. Also you can choose which role to notify on each node.

Drupal 8 - How building modules is different now

Drupal 8 is in alpha now, so its a good time to dip your feet in it and notice the major differences Drupal 8 introduced after adopting some Symfony low-level components such as HttpKernel, Routing, EventDispatcher, DependencyInjection, and ClassLoader, for a higher level changes see Hector Iribarne’s presentation on Drupal 8 changes.

In this blog post I will try to build a D8 module and walk through the basic steps.

In this example series we will develop a module and build on it during the coming blog posts.

Lets start.

In brief, our module is basically is a greeting box, visitors come to the site and leave you their greetings and comments, you as an admin can look at these greetings and delete the ones that you do not like. So the basic required architecture is: - A table in the database to hold the greetings. (Model or schema API) - Menu callbacks (Routes) - Controllers to handle the logic.

A good point to note here is that custom and contributed modules/themes now live inside /themes or /modules in the root folder, but still you can place them inside sites/all/modules or sites/all/modules themes.

Like any module in Drupal 7, modules in Drupal 8 need a .info file but this time in a YML format, so we create our hello.info.yml inside our hello directory that we create.

name: Hello
type: module
description: "Hello Drupal 8"
version: 1.x
package: Custom
core: 8.x

Before enabling the module, lefts define our database model, just as in D7 place our code in hello.install 

 

function hello_schema() {
  // Lets keep it basic here and only define the textarea
  $schema['names'] = array(
    'fields' => array(
      'name' => array(
        'type' => 'varchar',
        'length' => 40,
        'not null' => TRUE,
      )
    ),
    'primary key' => array('name'),
  );

  return $schema;
}

After enabling the module we will see the database table that we defined. Menu callbacks in D7 used to live in .module file but not anymore (will come to that later) after introducing Symfony’s routing system, so lets create a file and call it hello.routing.yml and tell Drupal that we need a page that holds a form for people to greet us. name_add: path: 'add/name' defaults: _form: '\Drupal\hello\Controller\AddName' _title: 'Add your name' requirements: _permission: 'access content' Basically telling our module to define a menu route ‘add/name’ that takes its logic from AddName controller, page title should be “add your name”, and anyone with “access content” permission should be able to access this page. As we said before, menu callbacks do not live in .module anymore, however, its a good practice to add a hook_menu() in the .module file function hello_menu() { $items['add/name'] = array( 'title' => 'Contact form categories', 'description' => 'Create a system contact form and set up categories for the form to use.', // point to name_add route in hello.routing.yml 'route_name' => 'name_add', ); return $items; } Now we should create the controller AddName that will handle displaying the form in the page callback, so create a directory inside the module hello

mkdir -p lib/Drupal/hello/Controller
cd lib/Drupal/hello/Controller
vim AddName.php

In AddName.php

namespace Drupal\hello\Controller;
use Drupal\Core\Form\FormInterface; 

class AddName implements FormInterface {
  function getFormID() {
    return 'add_name_form';
  }
  
  function buildForm(array $form, array &$form_state) {
    // The usual Drupal FAPI
    $form['name'] = array(
      '#type' => 'textfield',
      '#title' => t('Your name')
    );
    $form['actions'] = array(
      '#type' => 'actions'
    );
    $form['actions']['submit'] = array(
      '#type' => 'submit',
      '#value' => 'Add'
    );
    return $form;
  }
  
  function validateForm(array &$form, array &$form_state) {
    // Your validation code goes here
  }
  
  function submitForm(array &$form, array &$form_state) {
    $name = $form_state['values']['name'];
    drupal_set_message(t('Added %name as new participant.', array('%name' => $name)));
    // Remember: We should create a HelloStorage controller with the method add() later
    HelloStorage::add($name);
    // Redirect after form submission to add/name that or any other page that exists.
    $form_state['redirect'] = 'add/name';
  }
  
}

 

Now that we have our page add/name ready, lets create the HelloStorage controller before we try out the submission In the same directory that holds our controllers (lib/Drupal/hello/Controller) create the file HelloStorage.php 

namespace Drupal\hello\Controller;

Class HelloStorage {
  
  static function getAll() {
    return db_query('SELECT name FROM {names}')->fetchCol();
  }
  
  static function add($name) {
    db_insert('names')->fields(array('name' => $name))->execute();
  }
  
  static function delete($name) {
    db_delete('names')->condition('name', $name)->execute();
  }

}
}

Alright, the form is ready to work.

Now lets create the admin page that will display the greetings. First we need to define our route in hello.routing.yml

name_list:
  path: 'list/names'
  defaults:
    _content: '\Drupal\hello\Controller\ListNames::content'
    _title: 'People'
  requirements:
    _permission: 'administer greetings'

The route needs a controller to handle it, so in the controllers directory we create the file ListNames.php and we define content() method 
namespace Drupal\hello\Controller;

class ListNames {
  public function content() {
    $people = HelloStorage::getAll();
    $output = l('+ Add name', 'add/name');
    $output .= '<table><tr><td>Name</td><td>Delete</td></tr>';
    foreach ($people as $person) {
      $output .= '<tr><td>' . $person . '</td><td><a href="delete/' . $person . '">Delete</a></td></tr>';
    }
    $output .= '</table>';
    return $output;
  } 
}

As we can see from the code above, there is an l() function that create a delete link for the records in the database, that means we need to create a route for that, so in our routing file

 

name_delete:
  path: 'list/delete/{name}'
  defaults:
    _form: 'Drupal\hello\Controller\DeleteName'
    _title: 'Are you sure?'
  requirements:
    _permission: 'administer greetings'

Now we create our DeleteName controller 

namespace Drupal\hello\Controller;  

use Drupal\Core\Form\FormInterface;

 

class DeleteName implements FormInterface {
  function getFormID() {
    return 'delete_name_form';
  }
  
  function buildForm(array $form, array &$form_state) {
    // Just to confirm the deletion
    $form['actions']['submit']['yes'] = array(
      '#type' => 'submit',
      '#value' => 'Yes'
    );
    return $form;
  }
  
  function validateForm(array &$form, array &$form_state) {
    
  }

  function submitForm(array &$form, array &$form_state) {
    $name = arg(2);
    HelloStorage::delete($name);
    drupal_set_message(t('%name was deleted', array('%name' => $name)));
    $form_state['redirect'] = 'list/names';
  }

}

In the next post, we will build over our module to display the greetings in a block (Plugins) and define custom permissions.