{"id":239,"date":"2018-10-29T16:06:42","date_gmt":"2018-10-29T16:06:42","guid":{"rendered":"https:\/\/packlane.com\/blog\/?p=239"},"modified":"2018-10-29T16:06:42","modified_gmt":"2018-10-29T16:06:42","slug":"using-ecto-changeset-for-data-validation","status":"publish","type":"post","link":"https:\/\/packlane.com\/blog\/using-ecto-changeset-for-data-validation\/","title":{"rendered":"Using Ecto.Changeset for data validation"},"content":{"rendered":"\n
Ecto<\/code><\/a>, while commonly used as a database wrapper, can also serve as a powerful data validation tool thanks to its Ecto.Changeset<\/code><\/a>.<\/p>\n
Let’s say you have a list of movie data that you want to clean up and put into a nice %Movie{}<\/code> struct that would consist of the following fields:<\/p>\n
\ntitle<\/code>, a string,<\/li>\n
runtime<\/code>, an integer representing the movie runtime in minutes,<\/li>\n
cast<\/code>, a list of strings for each actor name,<\/li>\nreleases<\/code>, a map of the release dates for different countries.<\/li>\n<\/ul>\nWhile we could use Ecto’s schema\/2<\/code><\/a> macro to define this, we will instead use a simple struct¹ as we won’t actually be persisting this data to a database².<\/p>\n
defmodule Movie do\r\n  defstruct [:title, :runtime, :cast, :releases]\r\n\r\n  @type_constraints %{\r\n    title: :string,\r\n    runtime: :integer,\r\n    cast: {:array, :string},\r\n    releases: {:map, :date}\r\n  }\r\nend\r\n<\/code><\/pre>\nYou can check the list of available types here<\/a>, and if you can’t find what you need you can always define a custom type<\/a>.<\/p>\n
Now that we have defined our constraints, we want a function that’ll take a regular map, validate it and return a %Movie{}<\/code> if the input data is valid, or an error if it isn’t.<\/p>\n
The first thing we’ll do is filter out any field from the input map that isn’t part of our %Movie{}<\/code> struct or that can’t be converted to the right type. The cast\/4<\/code><\/a> function allows us to do just that:<\/p>\n
cast(data, params, permitted, opts \\\\ [])\r\n<\/code><\/pre>\ndata<\/code> is our container, so here we’ll be using an empty %Movie{}<\/code> struct³. Since we’re not using schema\/2<\/code><\/a> we’ll need to add the type constraints here too: {%Movie{}, @type_constraints}<\/code>. params<\/code> is the data we want to filter and validate, permitted<\/code> is a list of the keys we want to allow in params<\/code>. opts<\/code> allows to pass empty_values:<\/code> (for example you might want to set this to [[], “”] to make empty lists and empty strings nil) but we won’t be using it here. It defaults to empty_values: [\"\"]<\/code>.<\/p>\n
Let’s add a function to our Movie<\/code> module to create a new %Movie{}<\/code> from a map of params:<\/p>\n
defmodule Movie do\r\n  import Ecto.Changeset\r\n\r\n  ...\r\n\r\n  @allowed_keys Map.keys(@type_constraints) # We want to allow every key that has a type constraint\r\n\r\n  def new(movie_data) do\r\n    {%__MODULE__{}, @type_constraints}\r\n    |> cast(movie_data, @allowed_keys)\r\n  end\r\nend\r\n<\/code><\/pre>\nWe can now test it and make sure it works as expected:<\/p>\niex> Movie.new(%{title: \"The Hitchhiker's Guide to the Galaxy\", runtime: \"109\", cast: [\"Zooey Deschanel\", \"Martin Freeman\"], releases: %{\"UK\" => \"2005-04-28\", \"KR\" => \"2005-08-26\"}})\r\n#Ecto.Changeset<\r\n  action: nil,\r\n  changes: %{\r\n    cast: [\"Zooey Deschanel\", \"Martin Freeman\"],\r\n    releases: %{},\r\n    runtime: 109,\r\n    title: \"The Hitchhiker's Guide to the Galaxy\"\r\n  },\r\n  errors: [],\r\n  data: #Movie<>,\r\n  valid?: true\r\n>\r\n<\/code><\/pre>\nEcto uses %Changeset{}<\/code> to track changes in the data, which is really helpful when you want to persist your data to a database using Ecto or to get a list of validation errors, but not really useful for valid data in our case. Thankfully, apply_changes\/1<\/code><\/a> can help us with that:<\/p>\n
defmodule Movie do\r\n  ...\r\n\r\n  def new(movie_data) do\r\n    case changeset(movie_data) do\r\n      %{valid?: true} = changeset -> {:ok, apply_changes(changeset)}\r\n      changeset -> {:error, changeset}\r\n    end\r\n  end\r\n\r\n  def changeset(params) do\r\n    {%__MODULE__{}, @type_constraints}\r\n    |> cast(params, @allowed_keys)\r\n  end\r\nend\r\n<\/code><\/pre>\nThe %Changeset{}<\/code> struct has a valid?<\/code> flag that we can use to check if our data is valid. apply_changes\/1<\/code> will happily ignore it and return the underlying struct with our new data regardless of this flag⁴, so we have to check that it’s actually set to true<\/code> before calling it. Let’s try that again:<\/p>\niex> Movie.new(%{title: \"The Hitchhiker's Guide to the Galaxy\", runtime: \"109\", cast: [\"Zooey Deschanel\", \"Martin Freeman\"], releases: %{\"UK\" => \"2005-04-28\", \"KR\" => \"2005-08-26\"}})\r\n{:ok,\r\n %Movie{\r\n   cast: [\"Zooey Deschanel\", \"Martin Freeman\"],\r\n   releases: %{\"KR\" => ∼D[2005-08-26], \"UK\" => ∼D[2005-04-28]},\r\n   runtime: 109,\r\n   title: \"The Hitchhiker's Guide to the Galaxy\"\r\n }}\r\niex> Movie.new(%{runtime: \"not a number\", releases: %{\"UK\" => \"not a date\"}})\r\n{:error,\r\n #Ecto.Changeset<\r\n   action: nil,\r\n   changes: %{},\r\n   errors: [\r\n     releases: {\"is invalid\", [type: {:map, :date}, validation: :cast]},\r\n     runtime: {\"is invalid\", [type: :integer, validation: :cast]}\r\n   ],\r\n   data: #Movie<>,\r\n   valid?: false\r\n >}\r\n<\/code><\/pre>\nAs you can see, cast\/3<\/code> took care of converting runtime<\/code> to an integer and our release dates to %Date{}<\/code> structs<\/a>, returning a list of errors when some of the data was invalid.<\/p>\n
Now that we have our new\/1<\/code> function working, let’s add some validation!<\/p>\n
First, we want to make sure our movies always have a title and a runtime, so we’ll use validate_required\/3<\/code><\/a> to help us with that:<\/p>\n
defmodule Movie do\r\n  @required_keys [:title, :runtime]\r\n\r\n  ...\r\n\r\n  def changeset(params) do\r\n    {%__MODULE__{}, @type_constraints}\r\n    |> cast(params, @allowed_keys)\r\n    |> validate_required(@required_keys)\r\n  end\r\nend\r\n<\/code><\/pre>\nAnd that’s it! Ecto will return the changeset with valid?<\/code> flag set to false<\/code> if either of these two fields is missing in our data. There are a lot of validation functions in Ecto.Changeset<\/code><\/a> to add constraints on all sorts of data. For example, we could use validate_number\/3<\/code><\/a> to ensure the runtime<\/code> is always greater than 30 minutes and shorter than 5 hours, or validate_length\/3<\/code> to make sure there’s at least one actor in cast<\/code>.<\/p>\n
Sometimes, though, you’ll want to have validation rules that a specific to your use case. For this, Ecto provides an helper function validate_change\/3<\/code><\/a> that we can use to define custom validations.<\/p>\n
Let’s say we want to allow setting the IMDB page for each movie. Ecto doesn’t have a URL validator so we’re going to define a custom one and use it in our changeset\/1<\/code> function:<\/p>\n
defmodule Movie do\r\n  ...\r\n\r\n  defstruct [:title, :runtime, :cast, :releases, :imdb_url]\r\n\r\n  @type_constraints %{\r\n    ...\r\n    imdb_url: :string\r\n  }\r\n\r\n  ...\r\n\r\n  def changeset(params) do\r\n    {%__MODULE__{}, @type_constraints}\r\n    │> cast(params, @allowed_keys)\r\n    │> validate_required(@required_keys)\r\n    │> validate_imdb_url()\r\n  end\r\n\r\n  defp validate_imdb_url(changeset) do\r\n    validate_change(changeset, :imdb_url, fn :imdb_url, value ->\r\n      case URI.parse(value) do\r\n        %URI{host: \"imdb.com\"} -> []\r\n        %URI{host: \"www.imdb.com\"} -> []\r\n        _ -> [imdb_url: \"is not a valid URL\"]\r\n      end\r\n    end)\r\n  end\r\nend\r\n<\/code><\/pre>\nvalidate_change\/3<\/code><\/a> expects a changeset, the name of the field you want to validate and a function that will be passed the name of the field and its value and that should return a list of validation errors if any.<\/p>\n
Note that the custom validation function will only be called if there’s a change for the given field and if the new value isn’t nil<\/code>. Since we have set the type of imdb_url<\/code> to :string<\/code> the value passed to our validation function will always be a string.<\/p>\n
Let’s give it a try:<\/p>\n
iex> Movie.new(%{title: \"The Hitchhiker's Guide to the Galaxy\", runtime: \"109\", cast: [\"Zooey Deschanel\", \"Martin Freeman\"], releases: %{\"UK\" => \"2005-04-28\", \"KR\" => \"2005-08-26\"}, imdb_url: \"https:\/\/www.imdb.com\/title\/tt0371724\"})\r\n{:ok,\r\n %Movie{\r\n   ...\r\n }}\r\niex> Movie.new(%{title: \"The Hitchhiker's Guide to the Galaxy\", runtime: \"109\", cast: [\"Zooey Deschanel\", \"Martin Freeman\"], releases: %{\"UK\" => \"2005-04-28\", \"KR\" => \"2005-08-26\"}, imdb_url: \"https:\/\/www.not-imdb.com\/\"})\r\n{:error,\r\n #Ecto.Changeset<\r\n   action: nil,\r\n   changes: %{\r\n     cast: [\"Zooey Deschanel\", \"Martin Freeman\"],\r\n     imdb_url: \"https:\/\/www.not-imdb.com\/\",\r\n     releases: %{\"KR\" => ∼D[2005-08-26], \"UK\" => ∼D[2005-04-28]},\r\n     runtime: 109,\r\n     title: \"The Hitchhiker's Guide to the Galaxy\"\r\n   },\r\n   errors: [imdb_url: {\"is not a valid URL\", []}],\r\n   data: #Movie<>,\r\n   valid?: false\r\n >}\r\n<\/code><\/pre>\nEt voilà!<\/p>\n\nNote that you don’t actually need to define a struct and could just use a map instead.<\/li>\nschema\/2<\/code> will add a __meta__<\/code><\/a> field that’s mostly here to help with data persistance.<\/li>\n
We could also pass a non-empty struct, which is nice if you need to update existing data.<\/li>\n
apply_action\/2<\/code><\/a> would take care of this but requires an action<\/a> to be given, and that doesn’t make much sense in our case. You would probably want to use it when working with Phoenix form<\/a>, as they require an action to be set in order to display errors.<\/li>\n<\/ol>\nPS: Looking to join a collaborative remote engineering team writing Elixir and React? We’re often hiring, go check out our listings at https:\/\/packlane.com\/jobs<\/p>\n<\/body><\/html>\n","protected":false},"excerpt":{"rendered":"
Ecto, while commonly used as a database wrapper, can also serve as a powerful data validation tool thanks to its Ecto.Changeset. Let’s say you have a list of movie data that you want to clean up and put into a nice %Movie{} struct that would consist of the following fields: title, a string, runtime, an […]<\/p>\n","protected":false},"author":12357,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[9],"class_list":["post-239","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-engineering"],"_links":{"self":[{"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/posts\/239","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/users\/12357"}],"replies":[{"embeddable":true,"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/comments?post=239"}],"version-history":[{"count":0,"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/posts\/239\/revisions"}],"wp:attachment":[{"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/media?parent=239"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/categories?post=239"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/packlane.com\/blog\/wp-json\/wp\/v2\/tags?post=239"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}