AI in the Battle against fakes

Transcript

1. Henk Boelman Cloud Advocate @ Microsoft AI in the battle

   against fakes HenkBoelman.com @hboelman

2. Agenda Assignment & challenges Solution architecture Icon detection and validation

   with Custom Vision Logo classification with AMLS Recap and lessons learned

3. Assignment and challenges

4. Ask a sharp question

5. Is it a genuine or a fake package?

6. Collect the data

7. Can we have the data please? Sorry we don’t have….

   You are joking right?

8. 1 2 3

9. None
10. None

11. Break up the project and look at the smaller pieces

12. Solution Architecture Application flow and Azure design

13. Application AI Engine Management Front End

14. Ingredients Azure Durable Functions Docker Containers Azure Kubernetes Service Cosmos

    DB Azure Blob Storage Umbraco AMLS Cognitive Services Azure DevOps API Management

15. Umbraco config Create a product Tag product Upload Front image

    Run AI on image Push to engine

16. Umbraco config

17. AI Engine flow Photo of product Pre-Process image Match to

    Product in DB Text AI Logo AI Icon AI End evaluation

18. Icon detection and validation Application flow and Azure design

19. Object detection Classification Classification { Icon: Car Genuine: False, Score:

    95% }, { Icon: Truck Genuine: True, Score: 85% } Input Output Genuine: 2% | Fake: 95% Genuine: 85% | Fake: 3% Icon validator API

20. Label images Export Train Model Deploy Model Test Model Visual

    Object Tagging Tool Custom Vision Service Icon detection Run test-set against model

21. Label images Export Train Model Deploy Model Test Model Custom

    Vision Service Icon verification Custom Vision Service API Images in 2 folders Genuine / Fake Run test-set against model Image augmentation

22. Custom Vision Service ML lifecycle Data source Icon Validator API

23. Custom Vision Service An easy-to-use tool for creating your own

    custom image classifier
24. None
25. None
26. None
27. None

28. Demo: Custom Vision Service

29. Demo: Aka.ms/AI4DEV01-CustomVision

30. Logo classification A deep neural network to classify the Philips

    logo.

31. Computer Vision API Cut out PHILIPS Pre- Process Predict image

    Return Result Logo classification Icon Validator API

32. Label images Pre process Train Model Deploy Model Test Model

    Azure Machine Learning Service Logo classification Images in 2 folders Genuine / Fake Run test-set against model Image augmentation

33. Azure Machine Learning Services A fully-managed cloud service that enables

    you to easily build, deploy, and share predictive analytics solutions.

34. Is it Marge or Homer?

35. None
36. None

37. Prepare your environment Experiment with your model & data Deploy

    Your model into production

38. Step 1: Prepare your environment

39. Setup your environment VS Code Azure Notebooks Azure Portal

40. None

41. Datasets – registered, known data sets Experiments – Training runs

    Models – Registered, versioned models Endpoints: Real-time Endpoints – Deployed model endpoints Pipeline Endpoints – Training workflows Compute – Managed compute Datastores – Connections to data Azure Machine Learning Service

42. Create a workspace ws = Workspace.create( name='<NAME>', subscription_id='<SUBSCRIPTION ID>', resource_group='<RESOURCE

    GROUP>', location='westeurope') ws.write_config() ws = Workspace.from_config() Create a workspace

43. Create Compute cfg = AmlCompute.provisioning_configuration( vm_size='STANDARD_NC6', min_nodes=1, max_nodes=6) cc =

    ComputeTarget.create(ws, '<NAME>', cfg) Create a workspace Create compute
44. None

45. Step 1 Prepare your environment Create a workspace Create compute

    Setup storage

46. Step 2: Experiment with your model & data

47. Create an experiment exp = Experiment(workspace=ws, name=“<ExperimentName>”) Create an Experiment

48. Create a training file Create an Experiment Create a training

    file

49. Create an estimator params = {'--data-folder': ws.get_default_datastore().as_mount()} estimator = TensorFlow(

    source_directory = script_folder, script_params = params, compute_target = computeCluster, entry_script = 'train.py’, use_gpu = True, conda_packages = ['scikit-learn','keras','opencv’], framework_version='1.10') Create an Experiment Create a training file Create an estimator

50. Submit the experiment to the cluster run = exp.submit(estimator) RunDetails(run).show()

    Create an Experiment Create a training file Submit to the AI cluster Create an estimator

51. Create an Experiment Create a training file Submit to the

    AI cluster Create an estimator Demo: Creating and run an experiment

52. Azure Notebook Compute Target Experiment Docker Image Data store 1.

    Snapshot folder and send to experiment 2. create docker image 3. Deploy docker and snapshot to compute 4. Mount datastore to compute 6. Stream stdout, logs, metrics 5. Launch the script 7. Copy over outputs

53. Register the model model = Model.register( ws, model_name=‘My Model Name',

    model_path='./savedmodel', description='My cool Model') Create an Experiment Create a training file Submit to the AI cluster Create an estimator Register the model

54. Create an Experiment Create a training file Submit to the

    AI cluster Create an estimator Register the model Demo: Register and test the model
55. None

56. Demo: Aka.ms/AI4DEV02-amls-basics

57. Step 3: Deploy Your model into production

58. Developer Favorite Tools Version Control Build Pipeline Azure Container Registry

59. Project  App.py => run the model as an API

     Dockerfile => Docker Configuration  Requirements.txt => Python packages to be installed  aml_config/config.json => Azure Machine Learning config  downloadmodel.py => Build step to retrieve the latest model

60. Developer Favorite Tools Version Control Build Pipeline Azure Container Registry

61. Recap Azure Function for orchestrator Custom Vision for easy models

    Azure Machine learning Service Make services (API) for specific tasks / models GIT / CI / CD / Test everything

62. Lessons learned Wise lessons to take home with you …

63. You never have enough data Apply DevOps practices from the

    start… Break the problem into smaller pieces Think big, start small, grow fast

64. @hboelman github.com/hnky henkboelman.com Thank you! Read more on: henkboelman.com Aka.ms/AI4DEV01-customvision

    Aka.ms/AI4DEV02-amls-basics
65. None