15:02:34 <yamahata> #startmeeting neutron_northbound
15:02:34 <odl_meetbot> Meeting started Mon Jul 17 15:02:34 2017 UTC.  The chair is yamahata. Information about MeetBot at http://ci.openstack.org/meetbot.html.
15:02:34 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:34 <odl_meetbot> The meeting name has been set to 'neutron_northbound'
15:02:40 <yamahata> #chair mkolesni rajivk
15:02:40 <odl_meetbot> Current chairs: mkolesni rajivk yamahata
15:02:48 <yamahata> #topic agenda bashing and roll cal
15:02:51 <mkolesni> #info mkolesni
15:02:52 <yamahata> #info yamahata
15:02:59 <rajivk_> #info rajivk
15:03:01 <mbuil> #info mbuil
15:03:16 <yamahata> #link https://wiki.opendaylight.org/view/NeutronNorthbound:Meetings
15:03:38 <yamahata> any topics to discuss in addition to patches/buts?
15:03:51 <mkolesni> we can talk about the gate
15:04:00 <yamahata> espeically pike-3 is almost near, so we need to finally make decision on patches.
15:04:12 <yamahata> Yeah, gate is very unstable.
15:04:14 <mkolesni> also yamahata when will you know if you make it to ptg?
15:04:28 <yamahata> Not yet, hopefully in this week or next week.
15:04:48 <mkolesni> ok please let us know when you know concretely
15:04:52 <yamahata> sure.
15:05:01 <mkolesni> cool
15:05:25 <yamahata> anything else?
15:06:07 <yamahata> ok, move on
15:06:08 <yamahata> #topic Announcements
15:06:20 <yamahata> pike-3 is next week.
15:06:39 <yamahata> openstack cfp was closed.
15:07:15 <yamahata> ODL nitrogen karaf distribution is still unstable.
15:07:36 <yamahata> also ODL neutron northbound build is also unstable due to integration-test failure.
15:07:43 <yamahata> anything else to announce?
15:09:09 <yamahata> seems nothing else. move on
15:09:15 <yamahata> #topic Pike/Nitrogen planning
15:09:21 <yamahata> For pike, pike-3 is feature freeze.
15:09:26 <mkolesni> as pike-3 is next week id like to request that we progress with https://review.openstack.org/#/q/status:open+project:openstack/networking-odl+branch:master+topic:bp/dep-validations-on-create
15:09:29 <yamahata> We have three major issue.
15:09:58 <yamahata> Yeah, we're running out of time, we need to make decision.
15:10:14 <yamahata> For https://review.openstack.org/#/c/453581/ I don't see how we can come to consensus.
15:10:19 <yamahata> But we need to make progress.
15:10:28 <mkolesni> as we discussed you said you will give it another look
15:10:53 <mkolesni> we can progress and resolve bugs later
15:11:00 <yamahata> If we merge the patch, mkolesni  at lest I'd like you to commit to stabilize CI, tempest with v2driver.
15:11:18 <yamahata> Is it okay?
15:11:19 <mkolesni> im already working on stabilizing CI
15:11:37 <yamahata> Cool. then can you please rebase to resolve merge conflict?
15:11:40 <mkolesni> im investigating it further to understand whats going on
15:11:46 <yamahata> then we can make progress.
15:11:46 <mkolesni> yes ill rebase it
15:12:18 <mkolesni> also im reviewing the patches by rajivk_
15:12:20 <yamahata> Regarding to https://review.openstack.org/#/c/444648/ , have you looked at https://review.openstack.org/#/q/topic:bug/1683797 ?
15:12:33 <rajivk_> mkolesni, if some help is needed, please let me know.
15:12:51 <yamahata> then we can make the number of threads configurable and single timer in main process.
15:12:56 <yamahata> Does that work for you?
15:13:11 <yamahata> The series includes your patch, 444648.
15:13:33 <mkolesni> i didnt yet because i started investigating CI
15:13:41 <yamahata> Okay.
15:13:43 <mkolesni> but as this is bug fix it is not constrained by FF
15:13:48 <mkolesni> 0so not critical
15:14:11 <yamahata> cool.
15:14:57 <yamahata> Do you mean https://review.openstack.org/#/c/474851/ ?
15:15:49 <mkolesni> no this one has to get in by next week
15:16:39 <yamahata> I see.
15:16:57 <yamahata> another big patch is, https://review.openstack.org/#/c/465735/ dhcp port creation.
15:17:27 <yamahata> my understanding is, it can be merged (after several review if necessary.)
15:18:22 <yamahata> rajivk: has several bug fix patches, but they can be after pike-3.
15:18:34 <yamahata> any other patches for Pike-3?
15:19:43 <yamahata> seems nothing.
15:19:54 <yamahata> For Pike, another requirement is documentation migration.
15:20:13 <yamahata> https://review.openstack.org/#/c/483607/ is one of the patch
15:20:25 <yamahata> And I think it's good occasion to improve doc.
15:20:47 <yamahata> #link https://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html
15:21:06 <yamahata> #topic patches/bugs
15:21:26 <yamahata> we can discuss CI stuff.
15:21:30 <mkolesni> i rebased https://review.openstack.org/#/q/status:open+project:openstack/networking-odl+branch:master+topic:bp/dep-validations-on-create
15:21:35 <mkolesni> please review it
15:21:44 <yamahata> oh, how fast it is.
15:21:57 <yamahata> #action yamahata review https://review.openstack.org/#/q/status:open+project:openstack/networking-odl+branch:master+topic:bp/dep-validations-on-create
15:21:59 <mkolesni> ci stuff, so investigation revelased the problem with port status update which jhershbe fixed
15:22:10 <mkolesni> revealed
15:22:27 <yamahata> mkolesni: please go ahead.
15:22:40 <mkolesni> the problem is that ci jobs time out generally and then logs dont get collected
15:22:48 <mkolesni> so its hard to know what happened
15:23:08 <yamahata> when timeout occurs, logs aren't collected unfortunately.
15:23:20 <mkolesni> i have a "WIP" patch that reduces the timeouts in order to capture the logs but for some reason even that doesnt help sometimes
15:23:36 <yamahata> a bit before I thought it's collected. I suspect something changed with jenkins.
15:23:51 <mkolesni> maybe its something we can check on project-infra
15:24:11 <mkolesni> but im not that knowledgable about it so i dont know
15:24:43 <mkolesni> what i noticed now in some jobs theres a problem with the dhcp but i didnt manage to pin point it
15:24:51 <yamahata> Another possibility is, the ssh failures occur with specific tests. we can disable half of them.
15:24:52 <mkolesni> so im continuing investigation
15:25:11 <rajivk_> i noticed one thing.
15:25:13 <yamahata> I noticed that port binding of dhcp port sometimes fails at early tempest tests.
15:25:23 <mkolesni> i dont think its a good idea, the amount of tests failing is very high its not a few
15:25:33 <mkolesni> yes i saw that as well
15:25:36 <yamahata> It's because the first hostconfig update isn't done when subnet is created.
15:26:01 <mkolesni> i suspect its because of change of binding from topology to pseudo
15:26:20 <yamahata> network topology is also similar characteristic.
15:26:30 <yamahata> Anyway I'm thinking of BEFORE_CREATE/UPDATE of subnet.
15:26:47 <mkolesni> when binding was topology i think we had much less sporadic failures
15:26:49 <yamahata> if hostconfig isn't there, we can try to update hostconfig.
15:27:32 <mkolesni> what do u mean?
15:27:41 <yamahata> s/if hostconfig isn't there/if agentdb entry doesn't exist/
15:27:55 <mkolesni> but its part of provisioning
15:28:14 <mkolesni> maybe it needs to be done at some earlier stage in the provisioning
15:28:52 <yamahata> networking-odl retrieaves it into agentdb. If it didn't try to retrieve it, enforce networking-odl to retrieve it when port binding.
15:29:30 <mkolesni> thats a heavy solution
15:29:56 <mkolesni> im not sure its a good idea to have port binding stuck retrieving it
15:30:07 <yamahata> when port binding occurs, we know hostname, we can retrieve only single hostconfig.
15:30:15 <yamahata> Not entire list of hostconfig.
15:30:29 <mkolesni> i think in the set up of the ci the hostconfig needs to happen before the tempest run
15:30:50 <yamahata> And it's only before first full retrieval.
15:31:07 <mkolesni> i think in a real env this problem wont exist
15:31:21 <yamahata> In tempest case, we can fix it. But we can't enforce cloud admin to do that.
15:31:47 <mkolesni> if he uses pseudo agent he will run hostconfig before bringing up neutron
15:32:09 <mkolesni> then whether neutron or odl comes up first it should sync up when both are up
15:32:23 <mkolesni> not sure whats the current situation but thats the desired one IMHO
15:32:36 <yamahata> Hmm, are you saying we should introduce a utility to update agentdb based on hostconfig before running neutorn server?
15:33:11 <mkolesni> no im not familiar with it enough to know the solution
15:33:37 <mkolesni> wouldnt that be going back to the old way of preconfiguring everything in neutron?
15:33:54 <yamahata> hostconfig in ovs has to be populated.
15:33:55 <yamahata> No.
15:33:59 <yamahata> I don't get your question.
15:34:08 <yamahata> hostconfig in ovsdb needs to be populated.
15:34:12 <yamahata> That's right.
15:34:31 <mkolesni> im not really proficient in this area of the driver im afraid.. :/
15:34:42 <yamahata> The issue is that, networking-odl has to retrieve those value via ODL and then update agentdb
15:35:11 <yamahata> anyway I'll try to cook experimental patch and we'll see the outcome.
15:35:16 <mkolesni> ok
15:35:39 <mkolesni> ill keep digging to find the problems
15:35:57 <mkolesni> the amount of rechecks right now is too much
15:36:08 <yamahata> Yeah, the issue may reside in ODL netvirt. In that case, we can ask netvirt folks.
15:36:26 <mkolesni> no i see also failures in old netvirt
15:36:38 <mkolesni> which of course has bugs but these failures seem consistent
15:37:02 <yamahata> ohhh, it sounds like issues in networking-odl (or neutorn)
15:37:18 <yamahata> another issue is ODL L3 plug is broken for long time.
15:37:39 <mkolesni> i havent looked at that, do you have an idea whats wrong there?
15:38:06 <yamahata> yeah
15:38:25 <yamahata> It uses wrong db session
15:38:39 <mkolesni> you mean the deprecated one?
15:38:41 <yamahata> In the past, there was an attempt to fix it, https://review.openstack.org/#/c/356839/
15:39:14 <yamahata> After that, no one paid serious attention to it.
15:39:52 <mbuil> yamahata: does that mean that we should not use odl-router_v2 in the neutron service plugins?
15:39:56 <mkolesni> ill take a look at it i dont remember the details
15:40:28 <yamahata> mbuil: we should fix it at least.
15:41:08 <yamahata> The direction is to fix odl l3 or to switch l3 flavor(service driver)
15:41:38 <yamahata> and L3 plugin has been modified heavily.
15:41:54 <mbuil> yamahata: I thought after ODL Carbon release, that was the only way to do L3. I was told that the service_plugin=router (the one coming from OPenStack) was not working with ODL Carbon.
15:42:58 <yamahata> mbuil: Maybe that's right.
15:43:14 <yamahata> At least we know the current odl l3 v2 driver is broken. and it should be fixed for Pike release.
15:43:24 <mkolesni> i dont see any use of GUARD_TRANSACTION in neutron, please check with kevinbenton when will they remove it cause they might do it before pike release
15:43:59 <yamahata> Yeah, the l3 plugin code has been heavily modified by kevin.
15:44:26 <yamahata> midonet tried l3 flavor as experiment. he found that there are many thing to do.
15:44:39 <mkolesni> if they throw it out before pike is released well be in real sh*t
15:44:43 <yamahata> That's the right way in long term.
15:44:59 <mkolesni> can you check with kevin on that?
15:45:16 <yamahata> For Pike, we need to decide the direction.
15:45:46 <yamahata> that's obvious from git log.
15:45:57 <yamahata> Kevin has been worked on L3 issue since Ocata.
15:46:20 <yamahata> In Pike he reached the point that experimental midonet l3 flavor works somehow.
15:46:27 <mkolesni> not sure what you mean "that's obvious from git log"
15:46:53 <mkolesni> its obvious that "        # FIXME(kevinbenton): get rid of all uses of this flag
15:46:53 <mkolesni> "
15:47:00 <yamahata> #link http://lists.openstack.org/pipermail/openstack-dev/2017-July/119657.html
15:47:47 <mkolesni> seems he had no reply
15:47:52 <yamahata> Not yet.
15:48:23 <yamahata> Anyway ODL or midonet (or other out-of-tree) needs l3 flavor,.
15:48:34 <yamahata> So it is us (and yamamoto) to drive the discussion.
15:49:52 <yamahata> agent based L3 doesn't need it.
15:50:18 <mkolesni> maybe you should reply on the email so they know l3 is broken for us
15:50:53 <yamahata> Yeah, I wanted to see Kevin's reply first.
15:51:49 <yamahata> anything else to discuss?
15:51:54 <yamahata> mbuil: do you have any?
15:52:58 <mbuil> yamahata: well, just a question, I am trying to learn about odl-router_v2 and how to configure it when using port_binding_controller = pseudo-agentdb-binding
15:53:20 <mbuil> yamahata: I am not sure if there is a guide about that, the only thing I found is http://git.openstack.org/cgit/openstack/networking-odl/tree/doc/source/devref/hostconfig.rst
15:53:46 <mbuil> yamahata: apparently, it is important to add “bridge_mappings”: {“physnet1":"br-ex”} to the OVS config, but I am not sure if that is the only thing
15:53:50 <yamahata> For L3? just specify odl-router_v2 for l3 plugin
15:54:11 <yamahata> Nothing special in hostconfig for L3.
15:54:56 <yamahata> ODL L3 is mentioned because of future plan.
15:55:15 <yamahata> Right now you don't have to specify anything for L3 in hostconfig of ovsdb.
15:55:40 <yamahata> #topic open mike
15:56:01 <mbuil> yamahata: ok. and something I asked before and we were not sure. If I have two computes with OVS, should I have two ODL L2 agents when listing the network agents?
15:56:05 <yamahata> Probably we should improve the doc to avoid confusion.
15:56:17 <yamahata> Right.
15:57:07 <mbuil> ok, thanks about both questions. I will try to deploy l3 with ODL in the next days. In case I have Netvirt issues, do you know somebody who could help?
15:57:29 <yamahata> It's netvirt. #opendaylight-netvirt
15:57:35 <yamahata> you can find many there.
15:58:18 <yamahata> also there is mailing list.
15:58:22 <yamahata> https://lists.opendaylight.org/mailman/listinfo/netvirt-dev
15:58:30 <yamahata> you'd like to cc to odl-neutorn-dev too
15:58:40 <mbuil> yamahata: ok, thanks. I will update you guys with the progress
15:58:42 <yamahata> https://lists.opendaylight.org/mailman/listinfo/neutron-dev
15:58:53 <mbuil> yamahata: I am already in that list :)
15:59:09 <yamahata> Cool.
15:59:13 <yamahata> anything else?
15:59:53 <yamahata> thank you everyone.
16:00:02 <yamahata> #topic cookies
16:00:07 <yamahata> #endmeeting