Tag: attribute

  • Fixing issues after changing product attribute type from varchar to text

    Fixing issues after changing product attribute type from varchar to text

    In some cases there is a need to change the backend type of a catalog product attribute from varchar to text. The purpose of this change is to get more than 255 characters space for a string value.

    In this article I will cover the situation when problems occur after changing the backend type of an attribute.

    The Problem

    If the backend type of an attribute is changed, e.g. via install/upgrade script, Magento does not automatically copy and clean up old values. The consequence of that is that there are rudiments in the EAV value tables which cause some side effects. One of the side effects I was facing is editing a product which had a value for the affected attribute before the backend type change (in admin area). No values are displayed and there is no possibility to set a new value.

    So what to do if the change already happened and there is a mix between old value table rudiments and new value table entries?

    The Solution

    One possible solution to solve the issue are the following SQL Statements, here is an example for changing from varchar to text (you need to find out the id of the attribute from the eav_attribute table – here {attribute_id}):

    1. Copy the “value” from varchar table to text table for the case an entry for a product entity exists in both tables, but only if the “value” in the text table is null:

    UPDATE catalog_product_entity_varchar, catalog_product_entity_text 
    SET catalog_product_entity_text.value = catalog_product_entity_varchar.value 
    WHERE catalog_product_entity_varchar.attribute_id = catalog_product_entity_text.attribute_id 
    AND catalog_product_entity_varchar.entity_id = catalog_product_entity_text.entity_id 
    AND catalog_product_entity_text.store_id = catalog_product_entity_varchar.store_id 
    AND catalog_product_entity_text.entity_type_id = catalog_product_entity_varchar.entity_type_id 
    AND catalog_product_entity_text.value is null
    AND catalog_product_entity_varchar.attribute_id = {attribute_id};

     2. Copy entries which do not exist in text value table, but exist in the varchar table

    INSERT IGNORE INTO catalog_product_entity_text 
    (entity_type_id, store_id, attribute_id, entity_id, value)
    select entity_type_id, store_id, attribute_id, entity_id, value 
    from catalog_product_entity_varchar 
    where catalog_product_entity_varchar.attribute_id = {attribute_id} 
    and catalog_product_entity_varchar.value is not null;

    3. Delete entries from the varchar table

    DELETE FROM catalog_product_entity_varchar where attribute_id = {attribute_id};

    Important note

    Please verify the SQL, whether it is suitable for your purpose. Best practice is also to test it in a local / staging system and to back up the live database before applying the SQL on production. The solution is not perfect: I myself faced the issue, that the enterprise indexer cronjob took about 4h after applying the SQL, which blocked other cronjobs to be executed (about 50K products in DB). Possible way to avoid this is to separate “malways” (enterprise indexer) and “mdefault” cronjobs.

    I hope this  can be helpful. Feel free to comment if you faced this issue too or if you have any additions or a better solution.

  • How to Create a Category Image Attribute in Magento 2

    Creating a custom Category Attribute with an image upload is quite common feature requirement in our shops.
    So this blog-post will be about the steps you have to take to create a custom category attribute with an image upload in Magento 2.
    The post turned out to be quite long, but I wanted to provide a complete description of all steps necessary. So stay with us if your interested 🙂

    Our Module is called `Dev98_CategoryAttributes` and the image attribute will be called `dev98_icon`.

    Creating the Attribute

    In our projects we are creating the attributes using Install- or UpdateData classes, except when the attributes are explicitly managed manually.
    So to create the`dev98_icon` we might write something like this.

    $this->eavSetup->addAttribute(
        CategoryModel::ENTITY,
        'dev98_icon',
        [
            'type' => 'varchar',
            'label' => 'dev98 Icon',
            'input' => 'image',
            'sort_order' => 333,
            'source' => '',
            'global' => 2,
            'visible' => true,
            'required' => false,
            'user_defined' => false,
            'default' => null,
        ]
    );

    After running the setup:upgrade command to upgrade the application database and schema, we have created the attribute and should be able to store information for this attribute.

    Extending the Category Form

    To get an file upload for our attribute we need to extend the ui-component for the category form.
    The category form is defined in a category_form.xml, which can be extend by adding the following code to the file
    `Dev98/CategoryAttributes/view/adminhtml/ui_component/category_form.xml`

    <?xml version="1.0" ?>
    <!--
    /**
     * @copyright Copyright (c) 1999-2016 netz98 new media GmbH (http://www.netz98.de)
     *
     * @see PROJECT_LICENSE.txt
     */
    -->
    <form xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
        <fieldset name="dev98_attributes">
            <argument name="data" xsi:type="array">
                <item name="config" xsi:type="array">
                    <item name="collapsible" xsi:type="boolean">true</item>
                    <item name="label" xsi:type="string" translate="true">dev98 custom attributes</item>
                    <item name="sortOrder" xsi:type="number">100</item>
                </item>
            </argument>
            <field name="dev98_icon">
                <argument name="data" xsi:type="array">
                    <item name="config" xsi:type="array">
                        <item name="dataType" xsi:type="string">string</item>
                        <item name="source" xsi:type="string">category</item>
                        <item name="label" xsi:type="string" translate="true">dev98 icon</item>
                        <item name="visible" xsi:type="boolean">true</item>
                        <item name="formElement" xsi:type="string">fileUploader</item>
                        <item name="elementTmpl" xsi:type="string">ui/form/element/uploader/uploader</item>
                        <item name="previewTmpl" xsi:type="string">Magento_Catalog/image-preview</item>
                        <item name="required" xsi:type="boolean">false</item>
                        <item name="uploaderConfig" xsi:type="array">
                            <item name="url" xsi:type="url"
                                  path="dev98_category_attributes/category/categoryImageUpload/attribute_code/dev98_icon"/>
                        </item>
                        <item name="scopeLabel" xsi:type="string">[WEBSITE]</item>
                    </item>
                </argument>
            </field>
        </fieldset>
    </form>

    The above code block defines a new fieldset `dev98_attributes` which will contain one field`dev98_icon`.
    There is one parameter for the field `dev98_icon` I want to point out, which is the`uploaderConfig` array.
    In this array we have a parameter called url which defines the ActionController which is responsible for handling the image upload.
    As you can see we have provided a custom ActionController for handling the upload, because after taking a closer look to the Controller Magento uses for it’s category image upload, you will see why.
    The file is `\Magento\Catalog\Controller\Adminhtml\Category\Image\Upload`

    class Upload extends \Magento\Backend\App\Action
    {
        // […]
    
        /**
         * Upload file controller action
         *
         * @return \Magento\Framework\Controller\ResultInterface
         */
        public function execute()
        {
            try {
                $result = $this->imageUploader->saveFileToTmpDir('image');
    
                $result['cookie'] = [
                    'name' => $this->_getSession()->getName(),
                    'value' => $this->_getSession()->getSessionId(),
                    'lifetime' => $this->_getSession()->getCookieLifetime(),
                    'path' => $this->_getSession()->getCookiePath(),
                    'domain' => $this->_getSession()->getCookieDomain(),
                ];
            } catch (\Exception $e) {
                $result = ['error' => $e->getMessage(), 'errorcode' => $e->getCode()];
            }
            return $this->resultFactory->create(ResultFactory::TYPE_JSON)->setData($result);
        }
    }

    The Magento ActionController for handling the image upload is hardwired against the attribute `image`. There is no reasonable way we can reuse this code, so we will create our own in the next step.

    Image Upload ActionController

    Basically we have copied the code from Magento and refactored it to be more extensible:

    namespace Dev98\CategoryAttributes\Controller\Adminhtml\Category;
    
    use Magento\Framework\Controller\ResultFactory;
    
    class CategoryImageUpload extends \Magento\Backend\App\Action
    {
        // […]
    
        /**
         * Upload file controller action
         *
         * @return \Magento\Framework\Controller\ResultInterface
         */
        public function execute()
        {
            try {
                $attributeCode = $this->getRequest()->getParam('attribute_code');
                if (!$attributeCode) {
                    throw new \Exception('attribute_code missing');
                }
    
                $basePath = 'catalog/category/dev98/' . $attributeCode;
                $baseTmpPath = 'catalog/category/dev98/tmp/' . $attributeCode;
    
                $this->imageUploader->setBasePath($basePath);
                $this->imageUploader->setBaseTmpPath($baseTmpPath);
    
                $result = $this->imageUploader->saveFileToTmpDir($attributeCode);
    
                $result['cookie'] = [
                    'name' => $this->_getSession()->getName(),
                    'value' => $this->_getSession()->getSessionId(),
                    'lifetime' => $this->_getSession()->getCookieLifetime(),
                    'path' => $this->_getSession()->getCookiePath(),
                    'domain' => $this->_getSession()->getCookieDomain(),
                ];
            } catch (\Exception $e) {
                $result = ['error' => $e->getMessage(), 'errorcode' => $e->getCode()];
            }
    
            return $this->resultFactory->create(ResultFactory::TYPE_JSON)->setData($result);
        }
    }
    

    I left out the constructor and the `_isAllowed` method for a better readability.

    Our ActionController expects one argument `attribute_code` and hands that to the `imageUploader` which will handle the upload of the image.
    Now we have a reusable ActionController if we have to provide more custom category images.
    And as it turned out in one of our projects, we did need more than one.
    Furthermore we might extract that code into a vendor module.

    Current Status

    √ Create the Attribute
    √ Create category_form field
    √ Create the Controller handling the upload

    We have already done quite some work but we are not done yet.

    If you try running the code created so far you might see an Image Upload and the Image will be uploaded to the tmp folder `media/catalog/category/dev98/tmp/`.
    But it will remain there and there will be nothing saved to our `dev98_icon` attribute for this category.

    So why is that?

    The data for our `dev98_icon` field in the request is provided as an array. And as there is nothing changed regarding that fact, we end up with nothing being saved to our attribute.
    Futhermore, we need to handle the move from the tmp directory to the final image destination ourselves.
    That is OK in my opinion as there might be an error during the category save and we only want the image to be moved to its final destination when the save was successful.

    Category Save Observers

    To do so we need to observe the following two events:

    • catalog_category_prepare_save
    • catalog_category_save_after

    In our Observer for `catalog_category_prepare_save` we will convert the attribute image array to a string and in the `catalog_category_save_after` we will move the image from the tmp folder to its destination.

    <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Event/etc/events.xsd">
        <event name="catalog_category_prepare_save">
            <observer name="dev98_category_attributes_prepare_image_save"
                      instance="Dev98\CategoryAttributes\Observer\Category\CategoryImageDataPrepare" />
        </event>
    
        <event name="catalog_category_save_after">
            <observer name="dev98_category_attributes_save_after"
                      instance="Dev98\CategoryAttributes\Observer\Category\CategoryAttributesAfterSave" />
        </event>
    </config>

    `Dev98\CategoryAttributes\Observer\Category\CategoryImageDataPrepare`

    namespace Dev98\CategoryAttributes\Observer\Category;
    
    use Magento\Framework\Event\Observer;
    use Magento\Framework\Event\ObserverInterface;
    
    /**
     * Class ImageDataPrepare
     */
    class CategoryImageDataPrepare implements ObserverInterface
    {
        /**
         * List of available cms page image attributes
         *
         * @var []
         */
        protected $imageAttributes = [
            'dev98_icon',
        ];
    
        /**
         * Prepare image data before save
         *
         * @param Observer $observer
         *
         * @return $this
         */
        public function execute(Observer $observer)
        {
            /** @var \Magento\Catalog\Model\Category $category */
            $category = $observer->getCategory();
            $data = $observer->getRequest()->getParams();
            foreach ($this->imageAttributes as $attributeName) {
                if (isset($data[$attributeName]) && is_array($data[$attributeName])) {
                    if (!empty($data[$attributeName]['delete'])) {
                        $data[$attributeName] = null;
                    } else {
                        if (isset($data[$attributeName][0]['tmp_name'])) {
                            $data['is_uploaded'][$attributeName] = true;
                        }
                        if (isset($data[$attributeName][0]['name']) && isset($data[$attributeName][0]['tmp_name'])) {
                            $data[$attributeName] = $data[$attributeName][0]['name'];
                        } else {
                            unset($data[$attributeName]);
                        }
                    }
                }
                if (isset($data[$attributeName])) {
                    $category->setData($attributeName, $data[$attributeName]);
                }
            }
            if (isset($data['is_uploaded'])) {
                $category->setData('is_uploaded', $data['is_uploaded']);
            }
    
            return $this;
        }
    }

    The execute method looks rather complicated, but what it basically does is to set the image-name to the category model and mark the category as  `is_uploaded` so later-on we can detect we have to move an image. This code is mostly copied from the Magento Core, because as it turned out this part is also not reusable – yet.

    Current Status 2

    √ Create the Attribute
    √ Create category_form field
    √ Create the Controller handling the upload
    √ Observe the CategoryPrepareSave to get image name saved
    √ Observe the CategoryAfterSave to get image moved to destination

    But now we are done you might think.

    Well. No. There are still further steps to take to get this working completely.

    At this moment your image upload and saving will be working, but after refreshing the page you will see there is still something wrong as the image is either not shown at all or a placeholder image is shown.

    Plugin Category DataProvider

    After some debugging you will find out that there is class called  `\Magento\Catalog\Model\Category\DataProvider` which is used to provide the category data for the form.
    In this class we will find a method `getData` which does the image preparation for the standard `image`, again in a non extensible way:

    /**
     * Get data
     *
     * @return array
     */
    public function getData()
    {
        if (isset($this->loadedData)) {
            return $this->loadedData;
        }
        $category = $this->getCurrentCategory();
        if ($category) {
            $categoryData = $category->getData();
            $categoryData = $this->addUseDefaultSettings($category, $categoryData);
            $categoryData = $this->addUseConfigSettings($categoryData);
            $categoryData = $this->filterFields($categoryData);
            if (isset($categoryData['image'])) {
                unset($categoryData['image']);
                $categoryData['image'][0]['name'] = $category->getData('image');
                $categoryData['image'][0]['url'] = $category->getImageUrl();
            }
            $this->loadedData[$category->getId()] = $categoryData;
        }
        return $this->loadedData;
    }

    Next up we will create a Plugin that integrates after the  `getData` method and does the preparation for our custom attributes.

    Let’s create the `di.xml` with the Plugin class called
    `Dev98\CategoryAttributes\Plugin\Category\CategoryDataProviderPlugin`

    <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
        <type name="Magento\Catalog\Model\Category\DataProvider">
            <plugin name="dev98_categoryattributes_dataprovider_plugin"
                    type="Dev98\CategoryAttributes\Plugin\Category\CategoryDataProviderPlugin" />
        </type>
    </config>

    And finally our Plugin class itself:

    <?php
    /**
     * @copyright Copyright (c) 1999-2016 netz98 GmbH (http://www.netz98.de)
     *
     * @see PROJECT_LICENSE.txt
     */
    
    namespace Dev98\CategoryAttributes\Plugin\Category;
    
    use Dev98\CategoryAttributes\Api\CategoryUrlRepositoryInterface;
    use Magento\Catalog\Model\Category;
    use Magento\Catalog\Model\Category\DataProvider as CategoryDataProvider;
    
    /**
     * CategoryDataProviderPlugin
     */
    class CategoryDataProviderPlugin
    {
        /**
         * @var CategoryUrlRepositoryInterface
         */
        private $categoryUrlRepository;
    
        /**
         * CategoryDataProviderPlugin constructor.
         *
         * @param CategoryUrlRepositoryInterface $categoryUrlRepository
         *
         * @internal param StoreManagerInterface $storeManager
         */
        public function __construct(CategoryUrlRepositoryInterface $categoryUrlRepository)
        {
            $this->categoryUrlRepository = $categoryUrlRepository;
        }
    
        /**
         * AfterGetData
         *
         * we need to modify the data Array returned and generate the data array.
         * This includes the correct image url.
         * Without the information the Admin will not show this field due to an error
         *
         * @param CategoryDataProvider $subject
         * @param array $data
         *
         * @return array
         * @SuppressWarnings(PHPMD.UnusedFormalParameter)
         */
        public function afterGetData(CategoryDataProvider $subject, array $data)
        {
            /** @var Category $category */
            $category = $subject->getCurrentCategory();
            if (!$category) {
                return $data;
            }
    
            $attributeCodes = [
                'dev98_icon',
            ];
    
            foreach ($attributeCodes as $attributeCode) {
                $image = $category->getData($attributeCode);
                if (!$image) {
                    continue;
                }
    
                $imageName = $image;
                if (!is_string($image)) {
                    if (is_array($image)) {
                        $imageName = $image[0]['name'];
                    }
                }
                $categoryImageUrl = $this->categoryUrlRepository->getCategoryIconUrl($category, $attributeCode);
                $seoImageData = [
                    0 => [
                        'name' => $imageName,
                        'url' => $categoryImageUrl,
                    ],
                ];
                $data[$category->getId()][$attributeCode] = $seoImageData;
            }
    
            return $data;
        }
    }

    As you can see we extracted the url generation of the icon to a `CategoryUrlRepository` as this funcationality is need within more classes and in other modules as well.

    So last but not least here is the CategoryUrlRepository:

    <?php
    /**
     * @copyright Copyright (c) 1999-2016 netz98 GmbH (http://www.netz98.de)
     *
     * @see PROJECT_LICENSE.txt
     */
    
    namespace Dev98\CategoryAttributes\Repository;
    
    use Dev98\CategoryAttributes\Api\CategoryUrlRepositoryInterface;
    use Magento\Catalog\Api\Data\CategoryInterface;
    use Magento\Framework\Api\AttributeInterface;
    use Magento\Framework\UrlInterface;
    use Magento\Store\Api\Data\StoreInterface;
    use Magento\Store\Model\StoreManagerInterface;
    
    /**
     * CategoryUrlRepository
     */
    class CategoryUrlRepository implements CategoryUrlRepositoryInterface
    {
        /**
         * @var StoreManagerInterface
         */
        private $storeManager;
    
        /**
         * CategoryUrlRepository constructor.
         *
         * @param StoreManagerInterface $storeManager
         */
        public function __construct(StoreManagerInterface $storeManager)
        {
            $this->storeManager = $storeManager;
        }
    
        /**
         * @param CategoryInterface $category
         * @param $attributeCode
         *
         * @return string
         */
        public function getCategoryIconUrl(CategoryInterface $category, $attributeCode)
        {
            $url = '';
    
            $imageAttribute = $category->getCustomAttribute($attributeCode);
    
            if (!$imageAttribute instanceof AttributeInterface) {
                return $url;
            }
    
            $imageName = $imageAttribute->getValue();
    
            if (!$imageName) {
                return $url;
            }
    
            /** @var StoreInterface $store */
            $store = $this->storeManager->getStore();
            $baseUrl = $store->getBaseUrl(UrlInterface::URL_TYPE_MEDIA);
            $url = $baseUrl . 'catalog/category/dev98/' . $attributeCode . '/' . $imageName;
    
            return $url;
        }
    
    }
    

    That was the final class we needed to create to get this image upload working through out.

    Full Checklist

    Here is a short Summary of the classes and files created.

    √ Create the Attribute
    √ Create category_form field
    √ Create the Controller handling the upload
    √ Observe the CategoryPrepareSave to get image name saved
    √ Observe the CategoryAfterSave to get image moved to destination
    √ Create Plugin for Category\DataProvider to prepare Image Data for Frontend

    Final Thoughts

    If you have read this far, thanks for bearing with us on this one.
    We really spend some time debugging the different issues we had while creating the image upload.
    Multiple times we weren’t quite sure whether the error was in the javascript part or server-side.
    Sometimes we had to even debug javascript code to find out what was wrong with the implementation on the server-side, as there where errors that were not shown or the data was not provided as needed.

    But it was worth the effort and we know have a good glimpse of how the Category Admin is implemented.

    The code is still not perfect and might need some further polishing, but it should give you a good starting point.
    Right now I am thinking about putting together a public module on github to share this implementation.

    If you are interested in the full code of the module please let me know.
    And if there is an easier way to achieve these please let me know as well.

     

  • How to create attribute-options in Magento2

    Recently we had to create Configurable Products in our Product Import.
    To create those products we had to make sure that all simple products are generated before the configurable products and that all the attribute options for our configurable attribute are available.
    So this post will focus on how to create those attribute options and a problem where we had to spend some time figuring out.

    Preface

    We need to create those attribute options before we call
    $this->magentoImporter->validateSource($dataSource);
    otherwise the validation will fail because the options are validated as well.

    So let’s assume we have a collection of attribute-options that we want to ensure for an attribute.

    $attributes = [
        'dev98_diameter' => [
            '20mm' => '20mm'
            '30mm' => '30mm'
        ]
    ];

    Creating the Options

    We will iterate this array and generate an Option Data-Object for each of the attribute-options.
    This will be done using the Service Contract from Magento_Eav with the following classes:

    • Magento\Eav\Api\AttributeOptionManagementInterface (used to create the option)
    • Magento\Eav\Api\Data\AttributeOptionInterface  (one option)
    • Magento\Eav\Api\Data\AttributeOptionInterfaceFactory  (used as $this->optionFactory, to create an Option Data-Object)

    The resulting code might look something like this.

    foreach (attributes as $attributeCode => $options) {
        foreach ($options as $optionValue => $optionKey) {
            /** @var AttributeOptionInterface $option */
            $option = $this->optionFactory->create();
    
            $option->setLabel($optionValue);
    
            // Add Option
            $this->optionManagement->add(Product::ENTITY, $attributeCode, $option);
        }
    }

    After running this peace of code you will get attribute-options for your attribute.
    Note: This code does not check whether an attribute-option is already in the database.

    Running the Import

    Now when you start you product import using the Magento Core functionality your configurable products will be created and the simple items will be assigned properly.

    What I left out until now, was the trouble we had finding out how to create the option correctly.
    So the next chapter will be about the issue we face, how we solved it and based on that what not to do when creating an option.

    Problem with generating the Options

    At first we tried generating our options like this:

    $option = $this->optionFactory->create();
    $option->setValue($optionValue);
    $option->setLabel($optionValue);
    
    // Add Option
    $this->optionManagement->add(Product::ENTITY, $attributeCode, $option);

    As it turns this will fail with an Foreign-Key Constraint Violation while inserting the Option,
    basically saying the referenced option_id cannot be found.

    After some intense debugging I found out what was going on.
    Inside the OptionManagement the option->getValue()  is used to define the option_id if it is set.
    \Magento\Eav\Model\Entity\Attribute\OptionManagement::getOptionId

    private function getOptionId($option)
    {
         return $option->getValue() ?: 'new_option';
    }

    Debugging the Generation of an Option further we end up in the following method
    \Magento\Eav\Model\ResourceModel\Entity\Attribute::_updateAttributeOption

    protected function _updateAttributeOption($object, $optionId, $option)
    {
        $connection = $this->getConnection();
        $table = $this->getTable('eav_attribute_option');
        $intOptionId = (int)$optionId;
        
        // …
    
        if (!$intOptionId) {
            $data = ['attribute_id' => $object->getId(), 'sort_order' => $sortOrder];
            $connection->insert($table, $data);
            $intOptionId = $connection->lastInsertId($table);
        } else {
            $data = ['sort_order' => $sortOrder];
            $where = ['option_id = ?' => $intOptionId];
            $connection->update($table, $data, $where);
        }
    }

    This method gets the before mentioned option_id aka option->getValue()  injected, does an int-cast and based on that decides whether it is an update or insert.

    So if we come through this method with our sample having $optionId set to “20mm”
    the int-cast will return 20 and assume that the option is available and an update is to be made.
    That is the main reason for the mySQL Foreign-Key Exception.

    Why does it work with ‘new_option’?

    In case we leave out the $option->setValue()  in our implementation
    the optionId will be set to ‘new_option’ as shown above.

    The int-cast of the string ‘new_option’ will be 0 and therefore it will be detected as an insert.

    Conclusion

    It would have been great if Magento had some kind of validation as a first instance within the OptionManagement implementation, like throwing an Exception when the option->getValue()  is not an int. Or validating that the provided option_id is valid.

    Finally I want to point out, that you should NOT call $option->setValue(…)unless you have the exact option_id for this Option available.

    I hope this post is helpful and will save you some trouble when generating custom options programmatically.

    If you had similar experience or have something to add, feel free to leave a comment. Your feedback is always welcome.