Commit a141ab08 authored by Marco Pivetta's avatar Marco Pivetta

Merge pull request #706 from deeky666/DBAL-1023

[DBAL-1023] Fix line separators
parents 328153bd cb3bdbec
# Azure Federations # Azure Federations
Implementing Federations inside a new Doctrine Sharding Extension. Some extensions to the DBAL and ORM core have to be done to get this working. Implementing Federations inside a new Doctrine Sharding Extension. Some extensions to the DBAL and ORM core have to be done to get this working.
1. DBAL (Database Abstraction Layer) 1. DBAL (Database Abstraction Layer)
* Add support for Database Schema Operations * Add support for Database Schema Operations
* CREATE FEDERATION * CREATE FEDERATION
* CREATE TABLE ... FEDERATED ON * CREATE TABLE ... FEDERATED ON
* Add support to create a multi-tenent schema from any given schema * Add support to create a multi-tenent schema from any given schema
* Add API to pick a shard based on distribution key and atomic value * Add API to pick a shard based on distribution key and atomic value
* Add API to ask about federations, federation members and so on. * Add API to ask about federations, federation members and so on.
* Add Sharding Abstraction * Add Sharding Abstraction
* If a shard is picked via distribution key and atomic value fire queries against this only * If a shard is picked via distribution key and atomic value fire queries against this only
* Or query the global database. * Or query the global database.
2. ORM (Object-Relational Mapper) 2. ORM (Object-Relational Mapper)
* Federation Key has to be part of the clustered index of the table * Federation Key has to be part of the clustered index of the table
* Test with a pure Multi-Tenent App with Filtering = ON (TaskList) * Test with a pure Multi-Tenent App with Filtering = ON (TaskList)
* Test with sharded app (Weather) * Test with sharded app (Weather)
## Implementation Details ## Implementation Details
SQL Azure requires one and exactly one clustered index. It makes no difference if the primary key SQL Azure requires one and exactly one clustered index. It makes no difference if the primary key
or any other key is the clustered index. Sharding requires an external ID generation (no auto-increment) or any other key is the clustered index. Sharding requires an external ID generation (no auto-increment)
such as GUIDs. GUIDs have negative properties with regard to clustered index performance, so that such as GUIDs. GUIDs have negative properties with regard to clustered index performance, so that
typically you would add a "created" timestamp for example that holds the clustered index instead typically you would add a "created" timestamp for example that holds the clustered index instead
of making the GUID a clustered index. of making the GUID a clustered index.
## Example API: ## Example API:
@@@ php @@@ php
<?php <?php
use Doctrine\DBAL\DriverManager; use Doctrine\DBAL\DriverManager;
$dbParams = array( $dbParams = array(
'dbname' => 'tcp:dbname.database.windows.net', 'dbname' => 'tcp:dbname.database.windows.net',
'sharding' => array( 'sharding' => array(
'federationName' => 'Orders_Federation', 'federationName' => 'Orders_Federation',
'distributionKey' => 'CustID', 'distributionKey' => 'CustID',
'distributionType' => 'integer', 'distributionType' => 'integer',
'filteringEnabled' => false, 'filteringEnabled' => false,
), ),
// ... // ...
); );
$conn = DriverManager::getConnection($dbParams); $conn = DriverManager::getConnection($dbParams);
$shardManager = $conn->getShardManager(); $shardManager = $conn->getShardManager();
// Example 1: query against root database // Example 1: query against root database
$sql = "SELECT * FROM Products"; $sql = "SELECT * FROM Products";
$rows = $conn->executeQuery($sql); $rows = $conn->executeQuery($sql);
// Example 2: query against the selected shard with CustomerId = 100 // Example 2: query against the selected shard with CustomerId = 100
$aCustomerID = 100; $aCustomerID = 100;
$shardManager->selectShard($aCustomerID); // Using Default federationName and distributionKey $shardManager->selectShard($aCustomerID); // Using Default federationName and distributionKey
// Query: "USE FEDERATION Orders_Federation (CustID = $aCustomerID) WITH RESET, FILTERING OFF;" // Query: "USE FEDERATION Orders_Federation (CustID = $aCustomerID) WITH RESET, FILTERING OFF;"
$sql = "SELECT * FROM Customers"; $sql = "SELECT * FROM Customers";
$rows = $conn->executeQuery($sql); $rows = $conn->executeQuery($sql);
// Example 3: Reset API to root database again // Example 3: Reset API to root database again
$shardManager->selectGlobal(); $shardManager->selectGlobal();
## ID Generation ## ID Generation
With sharding all the ids have to be generated for global uniqueness. There are three strategies for this. With sharding all the ids have to be generated for global uniqueness. There are three strategies for this.
1. Use GUIDs as described here http://blogs.msdn.com/b/cbiyikoglu/archive/2011/06/20/id-generation-in-federations-identity-sequences-and-guids-uniqueidentifier.aspx 1. Use GUIDs as described here http://blogs.msdn.com/b/cbiyikoglu/archive/2011/06/20/id-generation-in-federations-identity-sequences-and-guids-uniqueidentifier.aspx
2. Having a central table that is accessed with a second connection to generate sequential ids 2. Having a central table that is accessed with a second connection to generate sequential ids
3. Using natural keys from the domain. 3. Using natural keys from the domain.
The second approach has the benefit of having numerical primary keys, however also a central failure location. The third strategy can seldom be used, because the domains don't allow this. Identity columns cannot be used at all. The second approach has the benefit of having numerical primary keys, however also a central failure location. The third strategy can seldom be used, because the domains don't allow this. Identity columns cannot be used at all.
@@@ php @@@ php
<?php <?php
use Doctrine\DBAL\DriverManager; use Doctrine\DBAL\DriverManager;
use Doctrine\DBAL\Id\TableHiLoIdGenerator; use Doctrine\DBAL\Id\TableHiLoIdGenerator;
$dbParams = array( $dbParams = array(
'dbname' => 'dbname.database.windows.net', 'dbname' => 'dbname.database.windows.net',
// ... // ...
); );
$conn = DriverManager::getConnection($dbParams); $conn = DriverManager::getConnection($dbParams);
$idGenerator = new TableHiLoIdGenerator($conn, 'id_table_name', $multiplicator = 1); $idGenerator = new TableHiLoIdGenerator($conn, 'id_table_name', $multiplicator = 1);
// only once, create this table // only once, create this table
$idGenerator->createTable(); $idGenerator->createTable();
$nextId = $idGenerator->generateId('for_table_name'); $nextId = $idGenerator->generateId('for_table_name');
$nextOtherId = $idGenerator->generateId('for_other_table'); $nextOtherId = $idGenerator->generateId('for_other_table');
The connection for the table generator has to be a different one than the one used for the main app to avoid transaction clashes. The connection for the table generator has to be a different one than the one used for the main app to avoid transaction clashes.
<?php <?php
namespace Doctrine\Tests\DBAL\Functional\Ticket; namespace Doctrine\Tests\DBAL\Functional\Ticket;
/** /**
* @group DBAL-421 * @group DBAL-421
*/ */
class DBAL421Test extends \Doctrine\Tests\DbalFunctionalTestCase class DBAL421Test extends \Doctrine\Tests\DbalFunctionalTestCase
{ {
protected function setUp() protected function setUp()
{ {
parent::setUp(); parent::setUp();
$platform = $this->_conn->getDatabasePlatform()->getName(); $platform = $this->_conn->getDatabasePlatform()->getName();
if (!in_array($platform, array('mysql', 'sqlite'))) { if (!in_array($platform, array('mysql', 'sqlite'))) {
$this->markTestSkipped('Currently restricted to MySQL and SQLite.'); $this->markTestSkipped('Currently restricted to MySQL and SQLite.');
} }
} }
public function testGuidShouldMatchPattern() public function testGuidShouldMatchPattern()
{ {
$guid = $this->_conn->query($this->getSelectGuidSql())->fetchColumn(); $guid = $this->_conn->query($this->getSelectGuidSql())->fetchColumn();
$pattern = '/[0-9A-F]{8}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[8-9A-B][0-9A-F]{3}\-[0-9A-F]{12}/i'; $pattern = '/[0-9A-F]{8}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[8-9A-B][0-9A-F]{3}\-[0-9A-F]{12}/i';
$this->assertEquals(1, preg_match($pattern, $guid), "GUID does not match pattern"); $this->assertEquals(1, preg_match($pattern, $guid), "GUID does not match pattern");
} }
/** /**
* This test does (of course) not proof that all generated GUIDs are * This test does (of course) not proof that all generated GUIDs are
* random, it should however provide some basic confidence. * random, it should however provide some basic confidence.
*/ */
public function testGuidShouldBeRandom() public function testGuidShouldBeRandom()
{ {
$statement = $this->_conn->prepare($this->getSelectGuidSql()); $statement = $this->_conn->prepare($this->getSelectGuidSql());
$guids = array(); $guids = array();
for ($i = 0; $i < 99; $i++) { for ($i = 0; $i < 99; $i++) {
$statement->execute(); $statement->execute();
$guid = $statement->fetchColumn(); $guid = $statement->fetchColumn();
$this->assertNotContains($guid, $guids, "Duplicate GUID detected"); $this->assertNotContains($guid, $guids, "Duplicate GUID detected");
$guids[] = $guid; $guids[] = $guid;
} }
$statement->closeCursor(); $statement->closeCursor();
} }
private function getSelectGuidSql() private function getSelectGuidSql()
{ {
return "SELECT " . $this->_conn->getDatabasePlatform()->getGuidExpression(); return "SELECT " . $this->_conn->getDatabasePlatform()->getGuidExpression();
} }
} }
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment