Developed with love by KnpLabs Hire us for your project!
35

AvroCsvBundle

by jdewit

A Symfony2 bundle for importing and exporting entities with CSV files

AvroCsvBundle Build Status

This bundle provides an easy way to upload data to your db using csv files with
just a few configuration parameters.

Status

This bundle is under development and may break.

Limitations

This bundle uses php and Doctrine2 and is not your best bet for
importing gargantuan csv files. Use your databases native importing & exporting
solutions to skin that cat.

Features

  • Import data by csv file
  • Export data to csv file
  • A few services for reading/writing csv files

Supports

  • Doctrine ORM

Installation

This bundle is listed on packagist.

Simply add it to your apps composer.json file

    "avro/csv-bundle": "*"

Enable the bundle in the kernel as well as the dependent AvroCaseBundle:

// app/AppKernel.php
    new Avro\CsvBundle\AvroCsvBundle(),
    new Avro\CaseBundle\AvroCaseBundle()

Configuration

Add this required config to your app/config/config.yml file

avro_csv:
    db_driver: orm # supports orm
    batch_size: 15 # The batch size between flushing & clearing the doctrine object manager
    tmp_upload_dir: "%kernel.root_dir%/../web/uploads/tmp/" # The directory to upload the csv files to
    sample_count: 5 # The number of sample rows to show during mapping

Add routes to your app/config/routing.yml file

AvroCsvBundle:
    resource: "@AvroCsvBundle/Resources/config/routing.yml"

Add the entities/documents you want to implement importing/exporting for

avro_csv:
    # 
    objects: # the entities/documents you want to be able to import/export data with 
        client:
            class: Avro\CrmBundle\Entity\Client # The entity/document class
            redirect_route: avro_crm_client_list # The route to redirect to after import
        invoice:
            class: Avro\CrmBundle\Entity\Invoice
            redirect_route: avro_crm_invoice_list

To exclude certain fields from being mapped, use the ImportExclude annotation like so.

namespace Avro\CrmBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Avro\CsvBundle\Annotation\ImportExclude;

/**
 * Avro\CrmBundle\Entity\Client
 *
 * @ORM\Entity
 */
class Client
{
    /**
     * @var string
     *
     * @ORM\Column(type="string", length=100, nullable=true)
     * @ImportExclude
     */
    protected $password;

Importing

Implement importing for as many entities/documents as you like. All you have to do is
add them to the objects node as mentioned previously.

Then just include a link to specific import page like so:

<a href="{{ path('avro_csv_import_upload', {'alias': 'client'}) }}">Go to import page</a>

Replace "client" with whatever alias you called your entity/document in the config.

Views

The bundle comes with some basic twitter bootstrap views that you can
override by extending the bundle.

Customizing each row

Want to customize certain fields on each row? No problem.

An event is fired when a row is added that you can tap into to customize each row of data.

Just create a custom listener in your app that listens for the 'avro_csv.row_added' event.

For example...

<?php
namespace Avro\CrmBundle\Listener;

use Symfony\Component\Security\Core\SecurityContextInterface;

use Doctrine\ORM\EntityManager;
use Symfony\Component\EventDispatcher\Event;

/**
 * Csv import listener
 *
 * @author Joris de Wit <joris.w.dewit@gmail.com>
 */
class ImportListener
{
    protected $em;
    protected $context;

    /**
     * @param EntityManager            $em      The entity manager
     * @param SecurityContextInterface $context The security context
     */
    public function __construct(EntityManager $em, SecurityContextInterface $context)
    {
        $this->em = $em;
        $this->context = $context;
    }

    /**
     * Set the objects createdBy field
     *
     * @param Event $event
     */
    public function setCreatedBy(Event $event)
    {
        $object = $event->getObject();

        $user = $this->context->getToken()->getUser();

        $object->setCreatedBy($user);
    }
}

Register your listener

services:
    import.listener:
        class: Avro\CrmBundle\Listener\ImportListener
        arguments: ["@doctrine.orm.entity_manager", "@security.context"]
        tags:
            - { name: kernel.event_listener, event: avro_csv.row_added, method: setCreatedBy }

Exporting

This bundle provides some simple exporting functionality.

Navigating to "/export/your-alias" will export all of your data to a csv and allow
you to download it from the browser.

If you want to customize data returned, just create your own controller action and grab
the queryBuilder from the exporter and add your constraints before calling "getContent()".

Ex.

    /**
     * Export a db table.
     *
     * @param string $alias The objects alias
     *
     * @return View
     */
    public function exportAction($alias)
    {
        $class = $this->container->getParameter(sprintf('avro_csv.objects.%s.class', $alias));

        $exporter = $this->container->get('avro_csv.exporter');
        $exporter->init($class);

        // customize the query
        $qb = $exporter->getQueryBuilder();
        $qb->where('o.fieldName =? 1')->setParameter(1, false);

        $content = $exporter->getContent();

        $response = new Response($content);
        $response->headers->set('Content-Type', 'application/csv');
        $response->headers->set('Content-Disposition', sprintf('attachment; filename="%s.csv"', $alias));

        return $response;
    }

To Do:

  • Allow association mapping
  • Finish mongodb support

Acknowledgements

Thanks to jwage's EasyCSV for some ground work.

Feedback and pull requests are much appreciated!

The MIT license

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is furnished
to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

avro_csv:
db_driver: orm
batch_size: 15
tmp_upload_dir: %kernel.root_dir%/../web/uploads/tmp/
sample_count: 1
objects:

# Prototype
object:
class: ~ # Required
redirect_route: ~ # Required
  • Merge pull request #13 from aur1mas/bugfix-delimiter
    By jdewit, 3 years ago
  • Merge pull request #12 from aur1mas/form-event
    By jdewit, 3 years ago
  • delimiter was not passed to Importer as 3rd param
    By aur1mas, 3 years ago
  • DataEvent is deprecated since version 2.1 and will be removed in 2.3. Code against \Symfony\Component\Form\FormEvent instead.
    By aur1mas, 3 years ago
  • fixed tests by removing unused code
    By aur1mas, 3 years ago
  • Added export event
    By jdewit, 4 years ago
  • added get composer
    By jdewit, 4 years ago
  • Added prefer source option
    By jdewit, 4 years ago
  • Add avro-case to kernel
    By jdewit, 4 years ago
  • Added multiple sample previews, revamped exporting, add some unit tests
    By jdewit, 4 years ago
  • fix typo
    By jdewit, 4 years ago
  • array type hint
    By jdewit, 4 years ago
  • escape assocations for now
    By jdewit, 4 years ago
  • add db_driver param
    By jdewit, 4 years ago
  • remove framework bundle annotations
    By jdewit, 4 years ago
  • removed jekyll
    By jdewit, 4 years ago
  • Merge branch 'release/realease'
    By jdewit, 4 years ago
  • Merge branch 'feature/preview-upload' into develop
    By jdewit, 4 years ago
  • Added row added listener, updated readme, added some tests, ...major clean up
    By jdewit, 4 years ago
  • add field mapping functionality
    By jdewit, 4 years ago
  • Change form name
    By jdewit, 4 years ago
  • Merge pull request #4 from krizon/config-fix
    By jdewit, 5 years ago
  • add default config nodes for import node by adding addDefaultsIfNotSet
    By krizon, 5 years ago
  • Merge branch 'master' of git://github.com/jdewit/AvroCsvBundle
    By jdewit, 5 years ago
  • update to 2.1
    By jdewit, 5 years ago
  • Merge pull request #3 from garak/patch-3
    By jdewit, 5 years ago
  • Merge pull request #2 from garak/patch-2
    By jdewit, 5 years ago
  • Update Util/Writer.php
    By garak, 5 years ago
  • Update Util/Writer.php
    By garak, 5 years ago
  • added support to add manyToOne relations via a legacyId
    By jdewit, 5 years ago